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ABSTRACT 

A comparison of multisite educational research 
projects involving gualitative research methods is reported. In phase 
1 of the project, telephone interviews with key staff members of 25 
projects determined ways in which each project handled important 
qualitative design and implementation choices. .Phase 2 consisted of a 
more intensive study of 5 projects: the Rural Experimental Schools 
Study, Experience-Based Career Education, Career Intern Program, 
Parental Involvement Study, and Dissemination Efforts Supporting 
School Improvement. Extensive interviews with key project members 
determined the project history, the interests of project staff and 
relevant outsiders, and methodological and administrative dilemmas 
characteristic of policy research. Telephone interviews were 
conducted with federal officials who monitored each project. 
Differences in the degree to which the methodologies of these 
projects were formalized were substantial. Formalization entails 
codification of questions and variables, standardization of data 
collection methods, and systematic reduction of verbal narrative to 
codes and categories. The variation results from different 
adaptations of social science to policy research and is more a 
function of the professional preferences of the research-team than of 
the technical requirements of policy research or the desire* of those 
who commissioned it. Appendices include profiles of the 2b multisite 
studies, tables, and an annotated bibliography^ (Author/KC) 
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MULTISITE QUALITATIVE POLICY RESEARCH IN EDUCATION: 
A STUDY OF RECENT FEDERAL EXPERIENCE 

(Contract No.* 400-80-0019) 
Abstract 



The last ^decade has seen the emergence in the policy arena 
of a new form % of qualitative research, -one intended to 
strenghten its ability to generalize while preserving in-depth 
description. These multisite qualitative studies address the 
same research question using similar data collection and- 
analysis procedures in a number of settings. 

, To gain systematic knowledge about this phenomenon a study 
was undertaken of policy research projects in the field of 
education that vere: 1) federally funded via a competitive RFP 
process/ 2) involved the application of qualitative method's of 
data collection and 3) compared several' research sites. 
Telephone interviews were conducted with a/key' staff member of 
25 such projects to learn how each project handled important 
--qualitative design and implementation choices. Subsequently a 
more intensive study of five projects was carried out to explore 
those choices in greater detail. Key documents from each " 
project were reviewed prior to. extensive interviews with project 
staff members at their offices. , The 'interviews focused on >the 
natural history of each project," the interests of project staff 
and relevant: outsiders, and methodological and administrative 
dilemmas characteristic of policy research. Later, telephone' 
interviews, were conducted with federal officials who monitored 
each project. 

One important distinction among thUse'mulitisite 0 
qualitative studies was the degree to which their methodology 
was , "formalized. * Formalization entails codification of the 
questions and variables to be studied, standardization of the 
data collection methods, and systematic reduction of"verbal 
narrative to codes and categories. The variation in 
formalization observed across projects was substantial and 
seemed to result from different adaptations of academic social 
science to the policy research context. Such variation was more 
a function of the 'professional preferences and behavior of the 
research team than of the technical requirements of policy 
research or the desires of those who commissioned it. 
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INTRODUCTION 



The tremendous growth of social programs in the 1960s led 
to even greater growth in the sponsorship of research about them 
in the 1970s. Initially, federally funded policy research 
concentrated on program outcomes, but gradually an expansion 
occurred to include an interest in program processes and 
implementation. Associated with this broadening of interest was 
a shift from the use of research designs that were exclusively 
quantitative to those that mixed quantitative and qualitative 
techniques and even to ones that were exclusively qualitative. 
Within the field of education this shift in research approach 
occurred in two phases. The earliest studies simply applied to 
multiple research sites qualitative approaches that had been 
developed within academia for the study of a single site. This 
was done in efforts to enhance generalizability without 
sacrificing in-depth description. In later studies, qualitative 
methods from academia were subjected to much modification as 
efforts were made to enhance their reliability. 

THE GENERALIZABILITY ISSUE 
After an initial attraction to qualitative approaches in 
their most highly developed academic form — that of 
anthropological ethnography— sponsors and researchers began to 
question the applicability of traditional ethnography to formal 
.program evaluations. A congressional aide, who later became an 
official at the National Institute of Education (NIE), reflected 
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on the early experiences of that agency with the sponsorship of 
multisite ethnographic studies and openly questioned their 
relevance to the immediate needs of policy makers (see 
Mulhauser, 1975) • However/ an experience ethnographer , who was 
serving as an advisor to one of NIE's contractors, argued that 
ethnography would lose its credibility as a form of scholarship 
if it attempted to be y^v^luative (see Wolcott, 1975). 

Concur r ent ivT ^uant i t a t i ve researchers questioned early 
efforts to generalize from qualitative studies (e.g./ see 
Campbell, 1974). Their concern was with the larger domain, if 
any, to which the findings from qualitative policy research 
could be applied. Often the question focused on the 
relationship of the sample under study to a larger population of 
policy interest (generalization from sample to population), but 
it also dealt with the relationship of what was being learned in 
individual sites to that at a "typical" site (generalization 
from case to sample). Moreover, participants in this debate 
were quick to note that such a concern about "statistical 
generalizability" failed to consider the fact that policy makers 
seldom were concerned solely about the effects of a given 
treatment on a specific population at one point in time.' Rather 
they were continually attempting to extrapolate from current 
experience to future aspirations. Cronbach (1982, p. 7i) , for 
example, argues that "The evaluation of a program /should lead/ 
to a statement about what to expect if a certain plan of action 
adopted (or continued) in a certain site or class of sites." 
Such forecasting required inferences that went well beyond 
statistical generalization — a problem not peculiar to 
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qualitative research. 

Traditional ethnography ignores issues of generalization 
and forecasting; it is radically particularistic. Spradley & 
McCurdy (1972/ p. 3) define ethnography as "the task of 
describing a particular culture* and differentiate it from 
ethnology which compares and explains. To Wolcott (1975/ p. 
112) "An ethnography is, literally/ an anthropologist's 
•picture 1 of the way of life of some interacting group. 11 Such a 
research tradition avoids ' efforts to explain/ generalize / or 
draw lessons for application in other settings. 

Sociologists doing qualitative research tended to be more 
willing to go beyond description. They had written widely on 
how qualitative research can be used to build theory/ including 
concepts arid explanations. Glaser and Strauss (1967)/ for 
example/ do not limit themselves to the study of a single 
well-defined "case." They and other qualitatively oriented 
sociologists proceed by generating explanations about a single 
social system intuitively and then disaggregating that case to 
individuals or events in order to seek confirmation or 
disconf irmation (Campbell/ 1975). 

One problem with this approach was that it threw very 
little light on generalizability beyond the particular case or 
on the conditions under which explanations derived from that 
case were likely to hold. All generalizations are, of course/ 
tentative. However/ one federal official suggested that the 
strenght of generalizations from case studies could be increased 
if many cases were included and the sample^met such criteria as 
substantial variety among cases, many similarities to the larger 



population of interest/ and few unique characteristics (Kennedy, 
1979). This line of reasoning was a major force behind the 
increasing federal interest in multicase qualitative studies 
during the 1970s. 

THE RELIABILITY ISSUE 
For qualitative research the issue of reliability involves 
the accuracy and stability of measurement. Quantitative 
researchers typically give great attention to objective 
measurement by carefully designing and documenting procedures 
and instruments (Selltiz, Wrightsman & Cook/ 1976). In 
traditional gualitatiave studies, there is less prespecif ication 
of data collection procedures in order to permit the reseasrcher 
to interact with the setting and gain insights in the process 
(Geer, 1967). This is one reason for the observation that 
qualitative research often increases construct validity at the 
expense of reliability (McGrath, 1982). Such a view does not 
imply that qualitative researchers were not concerned with 

accuracy. Rather they sought to achieve it through 

# .... 
nonquantitative means. These included extensive immersion in a 

setting, triangulation to check insights ?md hypotheses via 

multiple sources, socialization to a relativistic viewpoint and 

especially the habit of introspection to check against personal 

7 

bias (Wolcott, 1975). 

While the ethnographer's approach to accuracy is well 
accepted within the community of qualitative reseachers, it was 
not viewed positively by most quantitative researchers, at least 
not initially (see Campbell & Stanley, 1966). Moreover, it 
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encountered two problems in the policy world not faced in the 
academic one. The first stemmed from the adversarial nature of 
some policy research. Findings that are unpopular or 
disadvahtageous to an interest group are often attacked on 
methodological grounds. The researcher must be able to describe 
and defend data collection and analysis procedures, Furthe^ the 
data themselves may have to withstand extensive methdological 9 
critique and secondary analysis, as happened with two of the 
Coleman studies (see Hosteller & Moynihan, 1972; Hallinan & 
Olneck, 1982). Historically, qualitative researchers have had 
great difficulty disseminating their procedures and data in 
sufficient detail to make their studies amenable to either 
replication or secondary analysis. 

The second problem associated with the reliability issue 
was specific to multisite research. Given the many sites needed 
to increase generalizability, the researchers seemed to lose the 
flexibility of the single-site design. With more than one site, 
comparability of data collection/ reduction and analysis 
procedures across all sites tended to be given priority over 
in-depth description at individual sites. In general this was 
done .to ensure that whatever similarities and differences were 
noted among sites stemmed from inter-setting' rather than 
inter-researcher variation (Pelto & PeltO/ 1978). 

OVERVIEW OF THE REMAINDER OF THIS REPORT 

> 

" Questions about the generalizability and reliability of 
qualitative policy research greatly influenced the field in the 
1970s. The remainder of this report presents two papers 
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exploring those issues. The first draws upon a survey of 25 
multisite studies in the field of education and focuses 
primarily on the issue of generalizability (see Part II). It 
will be published in the February 1983 issue of Educational 
Researcher . The second paper examines in greater detail five 
studies in order to describe and explain variation in how they 
addressed the reliability issue (Part III). It is intended for 
publication in Evaluation Review . Technical details related to 
both papers are presented in an appendix (Part IV). 
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MULTISITE QUALITATIVE POLICY RESEARCH:' 
OPTIMIZING DESCRIPTION AND "3ENERAL I Z ABILITY 1 ' 

Robert E. Herriott and William A. Firestone* 

The classical qualitative educational research design is 
the case study. Studies of school life (Cusick, 1973; Wolcott, 

1973) , of the larger social forces affecting schooling (Ogbu, 

1974) , and of efforts to promote planned educational change 
(Smith & Keith, 1971) have used qualitative data in describing a 
single social setting. Typically, such studies emphasize 
in-depth description but provide a weak basis for generalization 
to orher settings. 

The last decade, however, has seen the emergence of a new 
form of qualitative research, one intended to strengthen its 
ability to generalize while preserving in-depth description. 
These multisite qualitative studies address the same research 
question in a number of settings using similar data collection 
and analysis procedures in each setting. They consciously seek 
to permit cross-site comparison without necessarily sacrificing 
within-site understanding. Although having some roots in 
academic social sciences (e.g., see Clark, 1970; Whiting, 1963; 
Whiting & Whiting, 1975), multisite qualitative research arose 

Robert E. Herriott is a research sociologist at 85 Jennie Dugan 
Road, Concord, MA 01742. William A. Firestone is Director of 
Field Studies, Research for Better Schools, 444 North Third 
Street, Philadelphia, PA 19123. 



13 16 



primarily in response to pressures from the federal government 
in the 1970s for studies that could overcome some of the 
weaknesses of large quantitative evaluations without being 
limited by the particularism of the single-site case study. 
Like many hybrids , it is today quite robust. However, these 
multisite qualitative studies were typically expensive endeavors 
and were done for specific policy purposes which the current 
federal administration seems neither to value nor to feel it can 
afford. 

There are two important reasons for reflecting on the 
historical development and potential utility of multisite . 
qualitative policy research at this time. Although it is 
unvelcomed by most social scientists, this current hiatus in 

y 

commissioning policy research at the federal level provides 
researchers and policy makers with an opportunity to consider 
these issues in some detail. Further/ the field of policy 
research has matured to the point where such considerations can 
b& very fruitful. In recent years qualitative researchers have 
moved beyond the need to* defend the legitimacy of their. craft in 
the policy arena (Rist, 1977;^ Smith, 1978; Stake, 1978). 
Moreover, quantitative researchers are 'beginning to acknowledge 
a role for qualitative research in policy and evaluation studies 
(Cronbach, 1982; Hoaglin, Light* McPeek, hosteller & Stoto, 

' 1*9 #2} ind \o consider the proper balance of qualitative and 

* ' • 

quantitative techniques (Cook & Reichardt, 1979? Smith & Louis, 
1982). In addition, practitioners 6f multisite qualitative 
policy* research now exhibit sufficient confidence in their 
draftsmanship to begin & process of public self-criticism with 
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an eye to improving their methods (Firestone & Herriott, 1982; 
Miles, 1979; Smith & Louis.- 1982; Yin, 1981). 

Efforts to examine multisite qualitative policy research 
suffer, however, from the absence of descriptive data about the 
field's status and growth. While there are useful first-person 
accounts of individual projects (e.g., see Fetterman, 1982) the 
field lacks systematic knowledge about a range of studies. The 
sections that follow offer a start in that direction. First we 
review the context and concern that led to the use of this 
innovative design in the 1970s and describe briefly an early 
effort to employ it in educational research. We then present 
the results of a formal survey of 25 educational studies using 
this general design to highlight some of its institutional and 
methodological features. Finally we speculate about the future 
of this design and suggest ways in which it might be refined by 
academically oriented educational researchers. 

HISTORICAL CONTEXT 
The 1960s were a period of dramatic growth in .federal 
funding of educational programs and the 1970s one of even 
greater growth in the sponsorship of research about them 
(National Institute of Education, 1976). Initially such 
federally funded policy research concentrated on the outcome of 
these programs, but gradually an expansion occurred from an 
exclusive concentration on pupil effects to an interest as well 
in program implementation and its contexts. Associated with 
this broadening in research focus was a shift in research 
designs from those that were exclusively quantitative to those 
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that were a mixture of quantitative and qualitative techniques 
and even ones that were exclusively qualitative. Although there 
was some advocacy in the 1970s for the use of better 
experimental designs in order to capture illusive program 
effects (see Bennett & Lumsdaine, 1975; Riecken & Boruch/ 1974)/ 
the general thrust of methodological innovation during that 
decade seems to have been to better understand and measure 
program implementation. A key component of this effort was the 
use of qualitative data collection/ reduction and analysis 
techniques. 

The reasons for this emphasis on qualitative designs have 
not been well documented/ but seem to involve a mixture of 
scientific and political "considerations focused on issues of 
political utility, design validity, and forms clearance. Quite 
prominent in the thinking of one federal official was a concern 
that the early evaluation designs were "findings poor" in the 
sense that they provided policy makers with .little understanding 
of why such programs as Head Start and Follow Through apparently 
had null effects or of how to improve them (Datta, 1982). A 
former federal official argues that the displeasure of policy 
makers with quantitative studies arose less from a concern about 
the lack of richness of their findings as from a concern about 
the findings themselves/ "few of which were liked by program 
advocates." To him qualitative studies were attractive to 
policy makers because "case study approaches * . > tend to yield 
less controversial findings/ ones with conclusions on both sides 
of a political decision" (Smith/ personal communication/ 1982). 

Regardless of where one stood on the political utility of 
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quantitative experimental studies there also seems to have been 
disagreement about the validity of their key dependent 
variables — measures of pupil performance. To many program 
advocates the then existing standardized tests mitigated against 
the ability of even the best quantitative designs to show 
positive effects for those minority group members who were the 
object of the most ambitious federally funded educational 
efforts (Cohen, 1975). Ethnographic studies of minority 
children vithin the complex cultural context of their families, 
neighborhoods and schools were often thought to be more likely 
to document beneficial program effects. . 

In addition to such concerns about the validity of 
dependent variables were questions about that of the independent 
variables, particularly the degree to which programs implemented 
with federal funds were sufficiently faithful to the intentions 
of their designers, or enacted in a sufficiently uniform manner 
across most sites, to permit a meaningful test of their effects 
(Weiss and Rein, 1970). One federal official responsible for the 
design and implementation of the Follow Through planned 
variation experiments increasingly came to favor more 
qualitative studies which could assist him and his colleagues in 
understanding why Follow Through "models" were apparently being 
implemented so variously in different schools (McDaniels, 
personal communication, 1982). 

A further contributor to the expansion from quantitative to 
qualitative policy studies seems to have been the growth of a 
cumbersome forms clearance process within the federal 
government. Although introduced originally to protect private 



industry from obtrusive and redundant federal data collection 
for regulatory purposes, forms clearance in the 1970s was 
embroiled in issues of federal-state relations and of individual 
privacy and became a major obstacle to standardized data 
collection efforts in * educational research (Datta, 1982). By 
the end of the decade it was rare to see a study of a federally 
funded educational program which did not at some point in its 
* design rely on unstandardized data collection procedures, often 

to minimize (or eliminate) the "forms clearance hassle . " 

In sum, concerns about excessive reliance on effects 
studies of questionable utility and validity, desires for more 
attention to program implementation issues and efforts to avoid 
the "forms clearance hassle" combined to create a broad-based 
demand for policy research modeled more after the traditonal 
case study than after the traditional experiment. 

THE RURAL EXPERIMENTAL SCHOOLS STUDY 
The Rural Experimental Schools Study was the first large 
scale, federally funded educational research effort to 
explicitly attempt the production of generalizable findings 
using traditional case study methods at multiple sites. The 
federal official responsible for its general design was openly 
skeptical of the utility of exclusively quantitative methods in 
program evaluation and argued actively and successfully for a 
strong qualitative component (Budding, 1972). Abt Associates 1 
proposal to study the program's ten rural sites adopted a 
two-pronged approach to optimizing description and 
generalizability — a series of quantitative cross-site studies 
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were to be carried out simultaneous with ten site-specific case 
studies. In describing the case study procedures, the project 
team placed great emphasis on the use of traditional 
ethnographic methods from the field of anthropology. 
Subsequently, experienced anthropologists and sociologists were 
recruited by Abt Associates to serve for approximately three 
years as full-time "on-site researchers" at each rural site 
(Herriott, 1977, 1982). 

The major product of these field workers was to be a case 
study of the site's educational change project in the context of 
its school system, community, and broader sociocultural context. 
After considerable negotiations among field workers, the 
project's administration, and representatives of the funding 
agency (the National Institute of Education), it was agreed that 
within broad guidelines, the case study authors would be free to 
let events at their sites be the primary determinants of case 
study content (Fitzsimmons, 1975). In addition the field 
workers were to prepare detailed social and educational 
histories f or^ each site, provide periodic qualitative reports on 
local events to those at Abt Associates' headquarters, and 
assist in the collection of structured data required for the 
quantitative cross-site studies. 

The adoption of the case study method in its classical form 
allowed these researchers to leam about its strengths and 
weaknesses in the policy context and to develop adaptations to 
enhance generalizability. The strength of the Rural 
Experimental Schools Study proved to be its ability to generate 
in-djipth description. The major product of the qualitative 
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segment of the study was eight book-length case studies of 
change projects (e.g., see Clinton/ 1979; Firestone, 1980). 

When the Rural Experimental Schools Study was commissioned 
little was known about how to use qualitative methodologies to 
develop general .conclusions and initially less effort was 
devoted to that task than to the production of site-specific 
narratives. Over time, however, two important steps were made 
in the direction of generalizability. One involved the 
incorporation of qualitative data into studies that were 
originally intended to be solely quantitative cross-site 
comparisons. Thus the cross-site survey of organizational 
change processes used reports from the on-site researchers as a 
major source of data for its key dependent variable — the degree 
of comprehensive change at sach site — as well as for insights 
about the nature of the change process in specific locations 
(Rosenblum & Louis, 1981). Equally important were efforts to 
synthesize insights from the case studies themselves to make 
generalizations about change processes in rural school 
districts. Limited by the decision to give the field workers 
autonomy with respect to the content of his/her study, these 
syntheses were carried out in two ways. One approach had the 
field workers write chapter-lenght case study narratives to a 
common outline and then asked outside experts to draw cross-case 
conclusions for policy makers and school administrators 
(Herriott & Gross, 1979). The second prepared more traditional 
"literature reviews" of the eight completed book-length case 
studies for policy makers (Herriott, 1980) and school 
administrators (Deal & Nutt/ ," 1979) . 
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A SURVEY OF MULTISITE QUALITATIVE STUDIES 
, The Rural Experimental Schools Study was part of the 
expansion in the use of multisite qualitative methods in the 
1970s. To learn more about this trend we undertook a survey of 
federal officials and qualitatively oriented researchers. 
Through a snowball sampling process wj identified 24 other 
projects, which: (1) were federally funded via a competitive RFP 
process, (2) involved the application of qualitative methods of 
data collection within at least a major part of the overall 
design, and (3) intended to compare two or more organizationally 
based research sites. 2 One of the most noticeable features of 
these projects is that whereas single-site case studies arise 
almost exclusively within academia most of these multisite 
projects were located within the type of applied social research 
firm which at that time was specializing in quantitative policy 
research. Overwhelmingly the qualitative studies were imbedded 
within larger (multistudy) projects having quantitative 
components as weil, and thus provided opportunities not only for 
cross-site qualitative, synthesis, but for the integration of 
qualitative and quantitative data. The funding for these 
projects was rather extensive (typically over one million 
dollars) and their duration lengthiy (typically at least two 
years) . J 

Four Design Issues 

While^the intent of multisite qualitative policy research 
is to optimize description and generalizability, there is a 
persistent tension between these two objectives which permeates 
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all research (Cook & -Campbell, 1979; McGrath, 1982). In 
multisite qualitative research this tension seems to revolve 
around four design issues. The 1 most prominent of these issues is 
the degree to whiich the data collection effort should be 
"structured" (Firestone & Herriott, 1982). Cross-site 
comparison and generalization require researchers at all. sites 
to use shared definitions of concepts and common data collection 
procedures to ensure that cross-site similarities and 
differences are characteristics of the sites and not the result 
of measurement procedures or researcher bias (Pelto & Pelto, 
1978). Yet such standardization encourages researchers to 
ignore the unique aspects of each site and to overlook processes 
and contexts that may make special contributions to the 
phenomena of interest. They also encourage the researchers to 
impose their definitions of the situation through premature 
conceptualization ( Blumer / 1969 ) . 

A high degree of structuring of data collection is obtained 
through the use of closed-end, precoded questionnaires and 
interview schedules. Unstructured modes of data collection 
include unobtrusive observation and sche'dule-f reie interviewing. 
These are the primary forms of data collection for most 
traditional case studies. Our snowball sampling process 
excluded projects that relied primarily on highly structured . 
data collection. Nevertheless/ when^ve examined the data 
collection procedures employed by a major qualitative study 
vithin each of these 25 projects, we were surprised to find that 
only five relied primarily on unstructured data collection 
techniques. The other 20 employed primarily a yariety of 
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semi-structured procedures, including site- visit guides which 
specify the questions that must be answered but not the specific 
data sources to be used, open-end interview guides, and 
instructions for focused observation. Such methods require that 
research issues be well thought out in advance rather than being 
derived "in the field. This heavy reliance on semi-structured 
procedures is clearly a major departure from the traditional 
single-site case study approach. It seems to represent an 
accomodation in the direction of quantitative methods, one made 
in order to facilitate cross-site comparison. / 

A second design issue concerns the number of sites to be 
studied. To a point, generalizability is enhanced by the 
inclusion of many sites (Kennedy, 1979). However, for any given 
budget level, increasing the number of sites limits the 
resources that are available for describing and analyzing events 
at any one site or for cross-site comparison. Within this 
sample, the fewest sites studied was three and the most was 60 
with a median of 11. The 25 studies seem to cluster into three 
distinct groups: those with three thru six sites (7 instances),, 
those with eight thru 22 sites (13 instances), and those with 30 
thru 60 sites (5 instances). The five studies with over 30 
sites r?ise an interesting question: how does one synthesize the 
mass of qualitative data from so^ many locations when attempting 
to draw generalizations? One risk in attempting such a 
cross-site- analysis is that the analyst will draw on the sites 
selectively, thus reducing data complexity but at the expense of 
representativeness. One alternative to such selectivity is to 
quantify the qualitative data through the use of rigorous coding 
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schemes so that formal statistical models can be used in 
carrying out the cross-site analysis. Yet such quantification 
can undermine the descriptive value of qualitative research that 
the multisite design is intended to exploit. 

A third issue is t^e length of time to be spent at each 
site for purposes of da-ta collection. Long-term immersion 
(generally of over one year) is the hallmark of classical 
ethnography (Wolcott, 1975) and is an important means of 
ensuring valid description (Dawson/ 1982). However, increasing 
the amount of time at any one site limits the resources 
available for studying other sites and for cross-site comparison 
and generalization. On-site presence in this sample of 25 
studies fell into three broad categories? one or two short 
visits to each site (10 instances)/ several intermittent vi&its 
(7 instances), and more continuous field work (8 instances). 

Finally, the research team can emphasize site-specific 
reporting/ as vas done in the Rural Experimental Schools Study, 
or cross-site, issue-specific reporting. Site-specific 

reporting is a literary device that enhances description but 

\ 

tends to mask similarities and differences across sites / thereby 
inhibiting generalization. Cross-site, issue-Specific reporting 
facilitates generalization, but often at the expense of 
site-specific context. Although most of the 25 studies we 
surveyed used both site-specific and cross-site qualitative 
reporting formats/ 12 emphasized the former and 13 the latter. 
Inter-issue Patterns 

In conducting this survey of multisite qualitative policy 
research projects we sought to uncover possible patterns in the 
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degree to which decisions affecting one of these four design 
issues are associated with decisions on the other three. For 
example, it seemed that the number of sites could well be an 
important factor in determining the degree to which report 
narratives emphasize site-specific phenomena, for there might be 
a point at which site-specific narrative becomes too cumbersome 
to utilize across all sites. Such a pattern does exist across 
these 25 projects. Whereas 6 of the 7 projects having six or 
fewer sites made extensive use of site-specific narrative, none 
of the 5 projects with 30 or more sites did so. However, when 
we looked for associations between gross categories of the. other 
five pairs of the four design decisions we could find none, 
suggesting little pervasive pattern in how the number of sites, 
the degree of on-site presence, the degree of emphasis upon 
unstructured data collection, and the degree of emphasis on 
wi thin-site narrative covary. 4 

The absence of patterning among all four design variables 
could simply reflect our inability to properly conceptualize or 
measure the most important choices faced by those responsible 
for these 25 studies, but we are inclined to view it as 
reflecting two other considerations. In particular it is 
important to keep in mind the very recent origin of this 
research form. During the past decade the logical connections 
among its various design options have been influenced by 
considerable experimentation on the part of both federal 
officials and researchers," as together they endeavored to 
understand better its strengths and weaknesses. However, it 
must also be kept in mind that such experimentation has taken 
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place within a context in which "the logic of science must come 
to terms with the logic of politics (Cronbach/ 1982/ p.ix)," 
Any policy research project arises within a unique political 
environment. Further each project is modified over time to 
reflect changes in that environment as they are interpreted by 
various federal officials and social scientists engaged in its 
design and implementation. Although policy researchers are 
clearly influenced by the logic of science/ their design and 
implementation decisions are not simply the result of the 
application of that logic to explicit a priori policy questions. 
Rather/ their decisions result from the interaction of political 
and scientific considerations from the time of the earliest 
conception of the need for research through the completion of 
the final report document. As a result no complex pattern of 
design and implementation choices can best satisfy the political 
requirements of more than a single project. 

THE FUTURE OF MULTISITE QUALITATIVE RESEARCH 
Multisite qualitative research flourished in the federal 
policy arena of the 1970s. It grew out of the qualitative 

w - 

traditions of academic social science and was modified to take 
into account the larger scale and more diverse audience of the 
policy arena. Unfortunately its future utility at the federal 
or state level or in academia is at present unclear. At the 
federal level/ the future of this research approach is enmeshed 
in debates about the government's responsibility for social 
action programs and for research. All of the 25 projects we 
surveyed were initiated to determine how the federal government 
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can intervene more effectively to improve service delivery at 
the state or local level, while the current administration 
clearly eschews such activism at the federal level as a matter 
of policy, it remains to be seen whether the federal government 
can fully divest itself of this responsibility. To the extent 
that it cannot— or to the extent that an administration oriented 
to state and local control chooses to fulfill its 
responsibilities by conducting research and demonstration 
programs to provide knowledge without imposing mandates for 
action— the use of multisite qualitative studies will continue. 
Also likely to facilitate the further use of this innovative 
method are sophisticated policy research efforts at the state 
level. The evaluation of the California School Imporvement 
Program, for example, involved extensive field work in 24 
schools in 14 districts (Berman, Weiler, Czesak, Gjelten & izu, 
1981). 

The current hiatus in the commissioning of large-scale 
policy research provides academically oriented educational 
researchers with unusual opportunities to experiment with this 
design, in the process multisite qualitative research will have 
to be adapted to the academic setting which, unlike the federal 
policy context, generally requires that research be done at more 
modest cost but with longer time lines. 

Academic researchers can facilitate the development of 
multisite qualitative research by addressing a broad range of 
methodological issues. For example, one current need is to 
understand better the consequences of /different staffing 
patterns for data collection. This is an especially important 
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issue in qualitative research where the researcher is often the 

» 

crucial ."instrument" (Sanday, 1979). It may be useful to 
conduct research which compares alternative data collection 
patterns f One such „ pattern is the use of a single investigator 
to carry out all- field work in, all sites (see Metz, 1978). Such 

an approach standardizes the data collection "instrument" across 

* 

sites without sacrificing the potential for in-depth 
description/ but it seems limited to situation involving no more 
than three or f.our sites. An alternative possibility is to 
provide greater data collection structure across multiple field 
workers, either through the use of field manuals (Campbell & 
Levine, 1973) or by having the different field workers prepare 
case szudy narratives for their sites using a common format 
agreed to after conducting some field work (see Herriott & 
Gross, 1979). Although there has been some effort to compare 
such approaches/ it has not been as systematic or as extensive* 
as it courd be (Perlman/ 1973). It would also be useful to know 
the conditions under which it is preferable ,to use "local 11 
residents or professional researchers as field observers and 
about the advantages and disadvantages of doing cross-si'tp 
comparison and generalization with field workers collaborating 
"in committee" or with "outside experts" who work only with the 
site-specific case study narratives. 

Another * crucial issuers the consequences of different 
approaches to the standardized reduction of unstandardized data. 
Such reduction is a necessary first step, to any analysis within 
or across sites (Goetz & LtfCompte, 1981).. The potential of any 
study for useful/ . valid description and generalization depends 
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on the analysts' ability to reduce data to a manageable form 
without distortion or loss* of meaningful detail. Studies with a 
large number of sites, or where the principal' investigator is 
not intimately familiar with all locations, are especially 
dependent on their approaches to data reduction. While we 
currently have some -craft discussion of how data reduction was 
done in specific projects, we need to know more about the 
advantages and disadvantages of the quantification of 
qualitative £ata (see Louis, 1982; Talmage & Rasher, 1981), and 
of verbal tabular and graphic data reduction devices (see 
Huberman & Miles, 1983; Smith & Nerenberg, 1981). Other issues 
in need of attention are the timing of site Visits in light of 
the phenomena under study, examination of processes and outcomes 
at different programatic levels (student, classroom, ^school, 
district, etc.) and alternative modes of presenting the results 
of research to policymakers. 

Due to the pressure of time, major methodological issues of 
, the type illustrated above can seldom be addressed 
systematically in the course of policy studies. . Academically 
oriented methodological studies represent an opportunity to more 
fully explicate the logic of this developing research form and 
to examine in detail its applicability in various policy 
contexts. 
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NOTES 



1. This paper has been prepared with support from The National 
Institute of Education under contract No. 400-80-0019 • It does 
not, however/ necessarily reflect the view of that agency. We 
are particularly indebted to Fritz Mulhauser of the Institute's 
staff for his unfailing facilitation of our research. 

2. The snowball sampling process .began with several highly 
visible qualitative researchers (Karen S. Louis, Matthew B. 
Miles, Ray C Rist, Robert Yin) and federal officials (Edward 
Glassman, Frederick Mulha.user, Marshall Smith, James Vanecko). 
Through their recommendations — and the recommedations of persons 
suggested by them — a roster of approximately 100 candidate 
projects was created. Subsequent telephone calls to a person 
more knowledgeable about each project led to the elimination of 
approximately 75 projects, in .most cases due to a failure to 
satisfy all three of the sampling criteria. For those projects 
meeting all criteria Arrangements were made for a one-hour 
telephone interview, generally with^the project's director. At 
the time of the interview the* inf drmaht was queried about 
his/her project using a highly-structured "project^profile" 
sheet as a guide. After the interview was finished a draft copy 
of the complete profile was sent to the informant and 
modifications requested if necessary. After the full set of 25 
profiles had been created and reviewed, they were used to code 
each project in terms of a series of summary categories. The 
key informants then reviewed that coding arid suggested whatever 
further modification of the profile sheets or summary tables 
seemed warranted. 

3. For a detailed description of each of the 25 projects, see 
Appendix A. 

4. For the basic tables supporting these conclusions, see 
Appendix C. 
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THE FORMALIZATION OF QUALITATIVE RESEARCH: 
AN ADAPTATION OF "SOFT" SCIENCE TO THE POLICY WORLD 1 

William a: Firestone ' and Robert E. Herriott ♦ 

A major development in evaluation and policy research in the 1970s was 
the introduction of qualitative ''techniques from anthropology, history, po- 
litical science, and' sociology (Cook & Reichardt, 1979).' However, \ the use 
of these techniques in the policy arena often differed greatly from that in 
the academic settings in which they originated. There seem to have been 
two reasons for this difference. The first was a concern for generaliza- 
bility and reliability that resulted from the early domination of policy 
research by quantitative economists and psychologists (e.g., see Riecken & 
Boruch, 1974) . The second was the need for qualitative researchers, like 
their quantitative predecessors, to respond to the requirements of rele- 
vance, timeliness, and utility of the policy arena (see Coleman, 1972). 

These two factors combined to produce a "formalization" of qualitative 
research having five major elements 2 , The first relates to^the intent of 
the inquiry. ^NJ 

1. Whereas traditional qualitative research tends to 

emphasize in-depth description, formalized qualitative 
research emphasizes explanation. 
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A second element regards the organizational form of the inquiry, 

2. Whereas traditional qualitative research tends to empha- 
size the conduct of inquiry by a single individual, 
formalized Qualitative research emphasizes the use of a 
tmiltipeVson team* 

The three additional elements have to do with the methodology of the in- 
quiry. — 

a 

3. Whereas traditional qualitative research, tends to 
emphasize the discovery of relevant questions and 
variables wl\ile in the.field^ formalized 'qualitative re- - * 
search emphasizes the codification of questions and 
variables before beginning fieldwork. 

Whereas traditional qualitative research tends tcf 
emphasize unstructured questioning, and observation, for- 
malized qualitative research emphasizes the standardi- 
zation of data collection procedures through the use of 
semistructured interview and 'observation protocols. 

5. Whereas • traditional qualitative research tends to 
emphasize extended presentation of verbAl narrative, 
formalized qualitative research .emphasizes the systematic 
reduction of verbal narrative to *codes and categories* 

While any one of these shifts, alone would constitute a minor adapta- 
tion to the policy arena, the simultaneous occurrence of all five has 
produced a radical transformation in the way qualitative research is con- 
ducted. This transformation has been driyen in part by the 'need to 
coordinate data collection in many sites and to ensure responsiveness to a 
client T s need for cross-site conclusions. In addition, some advocates of 
such coordination argue that problem-driven research using standardized 
techniques for data collection and analysis increases the truth or accuracy 
of qualitative research by responding to standards of validity and relia- 
bility traditionally associated only with quantitative research (Huberman & 

> +■ 

Miles, 1983). To them, what we have characterized as "formalization" 
represents a major improvement in the way that qualitative research is 
conducted and appraised. 
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- The advantages jnd disadvantages of this dramatic shift in the conduct 
of qualitative research are currently being debated by many of the princi- 
pals (see, e.g., Louis, 1982; Miles, 1979; Rist, 1980; Wolcott, 1980; Yin, 
1981). This paper does not enter that debate directly. Rather it seeks to 
inform it by examining in some detail five policy research projects. All 
five projects used research teams to carry out qualitative field work at 
multiple sites with the intent of making cross-site generalizations. They 
differed substantially, however, in the formalization of their methodologi- 
cal approaches. Our research suggests three competing explanations for 
this variation: the technical requirements of the research, the demands of 
the research sponsors, and the interests of the research teams and their 
professional networks. While concerns about methodological adequacy and 
policy utility create^ a broad interest in the formalization of qualitative 
research in the 1970s (see Herriott 4 Firestone, 1983), decisions about how 
much formalization would occur in specific instances seem to reflect the 
preferences of the research team more than any other factor. 

To date variation on grounds of personal and professional preference 
has provided a useful way for policy researchers and their 'federal sponsors 
to experiment with alternative approaches. However, the field of policy 
research currently needs stronger guidance on how much formalization of 
qualitative research is. appropriate under various technical circumstances. 
Such guidance is likely to be most useful if the degree of formalization 
thought to be most appropriate for particular research purposes is linked 
to realistic assumptions about the scientific and political requirements 
that qualitative policy research must meet (Cronbach, 1982). It is to 

« 

those requirements, and how they were addressed in the 1970s, that we now - 
turn. 
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A STUDY OF MULTISITE STUDIES 

Our understanding of formalization in qualitative research comes from 
a study of five multisite policy research projects in the field of educa- 
tion. We began with a "snowball" sampling process whereby we asked federal 
officials and educational researchers to help us identify projects which: 
(1) were federally funded via a competitive "request for proposals" (RFP) 
process, (2) involved the application of qualitative methods of data col- 
lection within at least a major part of the overall design, and (3) in- 
tended to compare two or more research sites* After identifying 25 such 
projects, we conducted telephone interviews with a key staff member 
(generally the director) in each. Our examination of those projects showed 
a great deal of variation in the standardization of data collection through 
the use of structuring devices* However, the apparent absence of associa- 
tion between the use of these devices and other variables raised a number 
of questions about how multisite qualitative studies are designed and 
implemented (Herriott and Firestone, 1983). 

To learn more about the dynamics of the research process, we initiated 

our own multisite study of five projects. All 25 projects were arrayed in 

terms of two variables thought to be associated with formalization: the 

number of sites and the length of time spent in collecting data at each 

site. Finally we selected for intensive study one project from each of the 

3 

five cells where either variable was relatively high (Table 1). 
The five projects are: 

* The Rural Experimental Schools (RES) Study. Initiated in 
1972, this complex multimethod project at Abt Associates 
Inc. explored the utility of comprehensive change efforts 
for reforming schools. In one of its five major substudies, 
ethnographic field work was conducted in 10 rural school 
districts over a three-year period by full-time "on-site 
researchers" trained in the discipline of anthropology or 
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Table 1 
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Distribution of 25 Studies by Length of Time 
on Site and the Number of Sites. (The five studies 
selected for intensive 3tudy are identified parenthetically.) 






Time on Site 








2-5 6-15 


16+ 






One or two short 

visits 2 3 


5 (EBCE) 






Several intermittent 

^its 1 5 (DESSI) 


0 






Many repeated visits 

or continuous presence 3 (CIP) 3 (RES) 


3 (PI) 




1 


* 

Note: For the identity of all 25 studies in the survey saxnple. 
Appendix A* 
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sociology. The field work was coordinated by Stephen J* 
Fitzsimmons, Robert E. Harriott and Michael B. Kane. 

The Experienced-Based Career Education (EBCE) Study. This 
research by The Huron Institute was inaugurated in 1976 to 
learn if JSBOL^lodca^ educa- 
tional laboratories would be effective when exported to a 
wide variety of public school settings. Attention was also 
given to learning about program implementation as a social 
♦process. Over a tfcree-year period three social scientists 
made several short visits to 45-schools. -The--amount-of time 
spent at each school site varied from one to 22 person days. 
Field work was conducted by Peter Cowden, John DeSanctis 
and Eleanor Farrar with David Cohen serving as senior 
advisor. 

The Career Intern Program (CIP) Study. The CIP program or- 
iginated at one site as a promising way to train minority 
youth to be employable workers or enter higher education. 
In 1978 it expanded to four geographically scattered sites. 
Through a multimethod study the KMC Corporation investigated 
what happens when an attempt is made to replicate the pro- 
totype in new settings, what produces "successful" program 
outcomes and what those outcomes were. For purposes of the 
'ethnographic substudy, approximately seven rounds of two- 
week visits were made to each site by a trained anthro- 
pologist. Key senior staff members included David 
Fetterman, Peter Treadway and Kasten Tallmadge. 

The Parental Involvement (PI) Study. Begun in 1978, this 
large-scale project conducted by System Development Corpora- 
tion described the form and extent of parental involvement 
within four federal educational programs. Data were col- 
lected at 57 sites over a four-month period by halfrtime, 
on-site field researchers. The formal academic training of 
these field workers varied from the pre-bachelors to post- 
doctoral level. All field work was coordinated by a st;aff 
of social scientists which included Ward Keesling, Ralph 
Melarangno, Al Robbins and Allen Smith, each of whom played 
an active role in cross-site data analysis. 

The Dissemination Efforts Supporting School Improvement 
(DESSI) Study. This complex multimethod study was commis- 
sioned in 1978 to reconsider assumptions underlying federal 
dissemination strategies, to learn how school districts 
undertake- pTs^^^ ej^n^ifce^^ 
government should promote fidelity to externally developed 
program models or local adaptations. Under the direction of 
David P. Crandall, The Network .Inc. coordinated the work of 
a series of subcontractors, one of whom undertook case 
studies of 12 schools. Field work of approximately eight 
days per site was carried out over a three-month period by 
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^v^wE?* ^ C !! ael HubenMn - Ma «hew B. Miles and 
Beverly Taylor, with Huberman and Miles subsequently con- 
ducting the cross-site analyses. 

Three projects were supported by the National Institute of Education and 
two by the Office of Planning, Budget, and Evaluation in the Office (later 
Department) of Education. All- five were carried out by private corpora- 
tions, a situation characteristic of v federally funded policy research in 
the 197, s.- All were aultimethod endeavors (Louis, 1982) which included 
quantitative surveys in addition to the "case studies" we focused on. They 
ranged in duration from 33 months (PI) to eight years (RES) . (See Appendix 
D for an annotated bibliography of illustrative publications from each 
project.) 

To learn about these projects, we reviewed such documents as requests 
for proposals, the proposals themselves, assorted planning documents, final 
reports, and published books and articles. In four cases we conducted 
extensive interviews with key project staff at their offices to learn about 
things not apparent in the documents. The interviews focused on the natural 
history of each project, the interests of project staff and relevant out- 
siders, and a series of methodological and, administrative dilemmas that we 
anticipated would arise frequently in multisite qualitative policy re- 
search. We spent from six to twelve hours talking- with several members of 
each project team. (This step was not taken with the RES study since we had 
been members of its staff, Herriott as the project's director and Firestone 
as an on-site researcher.) For all five projects we later talked to at 
least one of the federal officials responsible for its monitoring to better 
understand the projects' history and obtain' a client. perspective. Our 
presentation of these five projects begins with a description of the extent 
to which its most qualitative component was formalized, it then considers 
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the extent to which such formalization was influenced by three sets of 
factors that might guide the design and implementation of policy research. 

FORMALIZATION IN QUALITATIVE RESEARCH 
Formalized qualitative research projects tend to have more codified 
research questions at the beginning, more standardized data collection 
procedures, and more systematic means to reduce verbal data to categories 
for analysis. Table 2 describes variation among our five projects in terms 
of each of these definitional elements. 

The classical qualitative research begins with only the t.ost tentative 
research problem, and the first days in the field become an important time 
for fleshing out an understanding of the phenomena of interest (Geer, 
1969). Formalized qualitative research begins with well specified con- 
ceptual models and uses early field work to refine the conceptualization 
and either to check the feasibility of questions or primarily to collect 

the necessary data. RES embraced the traditional ethnographic field work 

\ 

mbd^- most fully, delegating the task of designing case studies to the in- 

\ 

dividualxon-site researchers, each of whom was an experienced field worker. 
Thus, there was never a central guiding conceptualization for its qualita- 
tive research. The EBCE team reported to us that in retrospect they could 
see the seeds of their major findings in their earliest proposal— perhaps 
reflecting ideas that they had developed in doing other studies of imple- 
mentation—but neither they nor the CIP team developed any formal a priori 
conceptualization to guide the research-. J?I and DESSI operated very dif- 
ferently. One staif member from the PI team devoted the first few months 
of the project to generating a model which elaborated five dimensions of 
parental involvement; he devoted less attention to specifying its causes 
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Table 2. Descriptive Character! cation of Five Projecta on Bight Indicators of Formalization 
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and consequences. The DESSI team developed a comprehensive model of the 
major variables thought to affect educational change efforts and explicated 
34 research questions. The RES, EBCE & CIP teams used early field work to 
become grounded conceptually, with RES & EBCE making explicit reference to 
using the first year to develop their theory along the lines suggested by 
Glaser and Strauss (1967). The PI and DESSI teams moved more quickly to 
collecting the data called for by their conceptual models although those 
models were modified somewhat over time (see Table 2, Indicators 1 and 2) . 

Data collection techniques can vary on a continuum from unstructured, 
where researchers simply observe and ask questions, to highly structured, 
where closed-ended precoded instruments are used. Exclusive reliance on 
precoded instruments was eliminated because it was considered quantitative 
rather chan qualitative research. However, ^^hec^Ws-coasfeterable varia- 
tion over- the rest of the continuum. , The RES study fever had a centrally 
imposed structure fW qualitative data collection. However, over time. some 
of the onsite researchers became progressively more-structured in Iheir 
approach, but only one developed formal interview guides. In contrast such 
progressive focusing was the rule on EBCE and CIP. At the end of the first 
round of site visits the EBCE team took tim« to reassess its research ob- 
jectives and to write position papers. They used the insights gained from 
. that collective process to guide later field work. Field' work for CIP was 
done in seven rounds of site visits. What was learned in the first was 
checked later. PI and DESSI relied primarily on semi-structured guides. 
PI developed theirs before the field work from the a priori conceptualiza- 
tions. DESSI finalized forms after the first, brief round of site visits. 
In both cases field work was gt^red to completing those guides, and there 
was frequent monitoring by senior researchers on both teams to assure that 
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adequate data were collected to answer each question at each site (Table 2, 
Indicators 3 4 4). 

Data reduction is the task of condensing information about each site 
to manageable proportions, and it too can vary in its prespecification, 
with more standardized modes generally thought to facilitate cross-site 
analysis. RES essentially'left this task to the discretion of the indi- 
vidual onsite researchers, and no formalized procedures were used in CIP 
since a single field worker covered all four sites. /EBCE experimented with 
a number of techniques, including creating a three-ring binder for each 
site in which field notes were cut up and organized by standard topics, and 
the use of wall charts to pom-ay sites and topics in matrix form. In PI 
each field worker prepared a narrative summary of data and observations for 
his or her site. These were follow* by site-specific syntheses done by 
the central staff following a standard outline and using the summaries and 
various interview forms as data. Before completing its field work, the 
DESSI team generated "interim" summaries of some sites and a case study 
outline with detailed data displays including dummy tables and tentative 
causal flow charts. These wereisubsequently completed for each case (Table 
2, Indicator 5). «f 

Generally, traditional qualitative approaches show thei| rigor through 
extensive presentation of data close to its raw form while formalized 
qualitative approaches emphasize presenting primarily higher order data, 
one or more steps- removed fronrthe original field notes. RES reported its 
qualitative site data through book-length case studies (Document^). 5 
EBCK presented illustrative quotes and vignettes in the cross-site 
analysis, but the reader cannot form an understanding of any specific site 
(Document D) . CIP used a similar approach but present chapter-length case 

50 47 



studies of each site (Document H) . PI presented some site-specific 
vignettes, but displayed most of its data in extensive narrative -tables 
with variables as rows and sites as columns (Document J). DESSI prepared 
case studies which are available to interested reviewers, but its public 
document features summary graphic displays for specific sites that were 
distilled from field notes during case study development (Document 
M) —Table 2, Indicator 6. 

The credibility of cross-site qualitative analysis can often be in- 
creased by the use of explicit preplanned procedures, including rules and 
displays for coded data, and by intersubjective checks requiring that there 
be consensus within the research team on the accuracy of coding and analy- 
sis (Firestone and Dawson, 1982). RES did not use standardized procedures 
for cross-site analysis, but its use of multiple independent synthesizers 
of the case study narratives (and in one instance the simultaneous presen- 
tation of five* syntheses in a single report — Document B) enhanced the 
credibility of its approach. EBCE and CIP relied on a similar form of in- 
tuitive cross-site analysis, although only one synthesis <was done in each 
case. The use of a team of three researchers on the EBCE study provided 
some checks and created the opportunity for each researcher to have to £fc 
defend his or her conclusions. Teamwork was less evident in the CIP case, 
but the overall project director aggressively reviewed and challenged all 
reports. PI required that all conclusions be apparent in cross-site *\ 
analysis tables, and that both table entries and the overall patterns be 
^efended in formal analysis committee meetings organized by conceptual 
element and by program studied (Document L) . Within PI both attacks and 
defenses of conclusions were extremely spirited. The DESSI senior 
researchers developed complex and thorough procedures for sorting sites and 




variables and for displaying the results (Document 0) . They checked each 
other's work but not with the same degree of ope* review required by the 
group context of PI (Table 2, Indicators 7 and 8) . 

In order to summarize the narrative picture of these five projects we 
read across the eight rows of Table 2 several times to get a sense of the 
range of variation on each" indicator. We then read down each of the five 
columns to discover the modal tendency within each project. Although our 
original intent was simply to divide the five projects into two ordered 
categories (low formalization and high formalization) the data reflected 
three (low, moderate, and high). The RES study stayed close to the 
traditional ethnographic approach by delegating the data collection and 
case study writing to~^^ and by deenphasizlng 

standardized cross-site analysis. It was at the" low extreme DESSI and 
PI, with their early conceptualization, extensive instrumentation and 
standardized data reduction, analysis and reporting techniques, were at the 
opposite extreme. CIP and EBCE were intermediate (Table 2). 

ALTERNATIVE EXPLANATIONS FOR VARIATION IN FORMALIZATION 
In*attempting to understand variation in formalization we first re- 
viewed the extensive notes we had taken in reading the various project doc- 
uments and in talking with project staff members and their federal 
monitors. Through this review, we identified three possible explanations 
for why these five projects were formalized to different extents. One 
emphasized the technical requirements of the research, particularly the 
generic questions that led to commissioning the .study. A second emphasized 
the contractual relationship between the organization selected to conduct 
the research and its federal sponsor. The third emphasi 2 ed the 
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predilections of the research team as professionals within an occupational 
group. We then reviewed out field notes a second time for evidence of the 
power of each explanation and summarized our findings by project and ex- 
planation. In so doing we were implementing an approach to qualitative 
research not too different ,£rom that of the five projects whose methods we 
were endeavoring to study. 

Technical Requirements 

To consider whether technical factors affected study formalization, we 
examined two issues: study purposes and the number of sites studied. 
These five studies include three program evaluations and two more general 
studies* RES, EBCE, and CIF focused on specific programs which were funded 
by the same agencies funding the research. The EBCE and CIP RFPs ex- 
plicitly stated that the results would be used to plan the possible contin- 
^ation-of_those programs. DESSI and PI addressed general policy issues and 
cut across several program agencies. Their importance to the continuation 
of specific programs was less clear. One might expect more formalization 
within the program evaluations where procedures for drawing inferences 
could conceivably be more subject to attack by entrenched political in- 
terests,, than within the generic studies having more diffuse constitu- 
encies. In fact, the opposite was true in these five cases. DESSI and PI, 
the two generic studies, are also the moist formalized. Moreover, upon 
closer examination the distinction between generic studies and evaluations 
turns out to be more apparent than real. In the case of the PI study, the 
managers of the four programs being studied— ESEA Title I, Follow Through, 
the federal Bilingual Program (ESEA Title VII) , and the desegregation pro- 
gram (ESAA)— became concerned that the study might create a "horse race" in 
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which some would look better than the others. Thus, an evaluative 
dimension was perceived by program people even when it was not important to 
the research sponsors. The evaluative purposes of the two other studies 
proved irrelevant; decisions not to continue the Experimental Schools and ' 
EBCE programs were made in Washington well before their respective studies 
were completed. In those cases interest among members of the research 
team, and to a lesser extent among the research sponsors, turned from pro- - 
gram evaluation to what could be learned about general processes of program 
implementation at the local level. ~ 

A second technical issue is the number of sites. Studies with more 
sites might be expected to be more formalized,- either to increase compara- 
bility or to reduce data collection and analysis costs. Yet, as in our 25 
project survey (Harriott 4 Firestone, 1983), we found no clear relationship 
between formalization and the number of sites. The two projects with the 
largest number of sites-PI with 57 and EBCE with 45~were highly and 
moderately formalized respectively; the next two-DESSI with 12 and RES 
with 10-were at the high and low extremes; and CIP, the project with the 
fewest sites (four), was moderately formalized. Here too, -however, things 
are not as clear as they seem because the number of sites can easily 
change. The EBCE, DESSI and PI RFPs were, quite ambiguous on this point. A 
fourth-RES-assumed that five sites would be included, but before the 
study began a sixth was added. One year later, primarily for reasons un- 
related to study design, the federal Experimental Schools Program added 
four more sites, thereby doubling the original estimate. 

s The number of sites was also subject to more subtle redefinition. 
Thus, PI began as a single generic study with the same research questions 
and data collection procedures to be used at all 58 sites. However, 
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because of pressures from the four federal programs, it was agreed that an 
interprogram comparative analysis would not be conducted. Instead, four 
within-program analyses (each with about 15 sites) were carried out as 
parallel replications. Similarly, although the EBCE team visited 45 sites 
during the first phase of its three-phase data collection process, more 
time was spent at some than at others and many sites were not visited more 
than once. From one to 12 days were spent at different sites, with only a 
quarter of them visited for as many as seven days (Document C). As a 
result, the team reports that their theory generating work relied most 
heavily on about. 15 of the 45 sites. In sum, technical factors associated 
with either general research purpose or the number of sites seem to have 
little predictable influence on the formalization of research. A major 
reason is that (at least for these five studies) technical factors, both 
abstract and seemingly concrete, are subject to frequent redefinition. 

The Contractual Relationship 

The purchaser of research might be expected to have a major influence 

on its design. Such a view from the federal side is made explicit by Baker 

(1975, p. 209-210) who explains that: 

Quite often the [federal] agency... knows how it wants the 
study conducted in evaluation research. . . . Many [federal] 
applied research administrators push for such a detailed speci- 
fication of the problem and research design that the only im- 
portant question lef t* for the contractor is how much will it 
cost to carry out the agency 1 s plan. Jhe agency, knowing what 
it wants done and how it wants it done, is looking for a 
skilled staff to carry out its needs, not somebody else's de- 
sires. 

After looking closely at federal request for proposal (RFP) preparation, 
contractor proposal writing arid negotiation and the post-award administra- 
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tion for each project, we concluded that the federal sponsors rarely 
specified the degree of formalization of a study. 

Selecting a Research Contractor . If Baker's view is correct, the 
federal agency's RFP is a critical document. It sets out the research spe- 
cifications, leaving the proposers to compete to see who can best carry out 
those specifications. Yet, the RFPs for all five of these projects were 
mach less clear than Baker's statement implies. In many ways they were 
little more than a Rorschach test, an ambiguous stimulus that could gen- 
erate a broad range of design responses. Consider the views of one of the 
PI proposal writers who describes the methodology of the RFP as "funny." 

[The PI staff member] says that people were to go out to sites 
but the number of sites was unclear. They were to cotiLl some 
quantitative data, but they were also to do some interviews 
wxth key individuals in the district and with parents Sere 
was supposed to be a division between quantitative data and 

Jn^W^VV' W \ S l6ft C ° the ^°P°*« S to work out 
the balance on depth vs. breadth. Moreover, the proposers were 
t asked to discuss the tradeoffs. proposers were 

(from field notes of PI site visit) 
The PI team's response was based partly on an estimate of what the agency 
wanted, but it also considered what it was competent to do and what it was 
interested in doing. A similar situation appeared within the RES study. 
The RFP clearly asked for descriptive "documentation" of each local project 
and required that one full-time person be assigned to live at each site. 
However, the nature of the documentation required was unclear; proposers 
were encouraged to consider approaches from a variety of academic disci- 
plines Including anthropology, but also history and sociology. Moreover, 
the project director perceived a conflict between these ambitious require- 
ments for sound scholarship and the need (also specified in the RFP) to 
provide technical assistance to local districts-a need that would take the 
onsite researchers out of a nonparticipant researcher role. 
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In some cases, proposal writers tried to make an intelligent interpre- 
tation of RFP requirements. In other cases, such requirements were simply 
ignored. Thus, the DESSI RFP called for a "cross-sectional survey- 
interview" study; there was no hint of that being supplemented with an in- 
tensive qualitative case-study component. Such an activity was initiated 
solely by The Network in its successful proposal. In the EBCE case, the 
proposal writing team concluded that one of the key requirements in the RFP 
(a focus on the fidelity of program implementation at the local level) was 
ill advised in view of the then emerging findings of the Rand "Change Agent 
Study" on the importance of mutual adaptation (see Greenwood, Mann & 
McLaughlin, 1975)^ Sensing the conflict over the issue of fidelity — and 
believing that it did not make sense to pursue fidelity questions — they 
describe their proposal as "an intricate dance around" that issue. Al- 
though, after contract award, representatives of the federal sponsor con- 
tinued to want a "discrepancy study," none was ever carried out. 

There was only one case among the five we studied of an RFP clearly 
requiring something that the proposing team did not want to do where the 
funding agency successfully insisted that it be done. The CIP RFP required 
that one part of the study be "an ethnography," and the winning team did 
not know what ethnography was. How that issue was resolved to include an 
ethnographic component is described in the next section. 

Generally, the ambiguity of the RFP stems from the numerous opportuni- 
ties during the RFP development process for a variety of individuals to add 
pieces. An interest in building broad support for the procurement within 
the agency leaves unresolved contradictions in the RFP document. For 
instance, the federal official responsible for the preparation of the PI 
RFP had "been hearing about ethnography and other impressionistic 
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approaches. 11 He saw the PI study as "a chance to break out and do a less 
quantitative, more qualitative study. 11 However, according to him, the per- 
son with whom he was collaborating in preparing the RFP "was more oriented 
towards a survey—one with questionnaires or checklists— and tried from 
time to time to nudge the RFP back towards a more quantitative approach." 
Thus, it is not surprising that the methodological requirements in the PI 
RFP seemed "funny" to the proposal writers; they were developed by two 
people of competing methodological preferences. 

Moreover, within the studies we examined, specific methodological ex- 
pertise was never the most important factor in the agency's decision 
regarding which proposal to fund. Often the federal sponsors were pri- 
marily concerned with the known competence of the proposed staff. The 
initial RES project monitor explained that: 

It boiled down to a judgment on the nature of the staff and 
their orientation to the study, . . . Substantively, the pro- 
posal was on target. It was interesting and we were favorably 
inclined, but we were skeptical about what a proposal is worth. 

To the PI project monitor management skills were more important than 

methodological orientations. He was looking for a "contractor who had the 

resources to have a lot of people out in the field at one time," one who 

could "get things done on time and within budget." Since the tasks in the 

contract's scope of work statement were acknowledged to be "less well 

defined" than was typically the case for that agency, it was anticipated 

that "the contract could easily get out of hand unless you had a strong 

contractor." 

Our data clearly suggest that the procurement process is seldom a cal- 
culated search for the contractor with the best design to achieve prespeci- 
fied federal objectives. It has been suggested that "it's more like a 
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pickup at a singles bar, with both parties wondering what will happen if 
they leave together/ 1 

Monitoring the Contractor . If federal influence is so limited during 
the selection process, will the agency perhaps have a greater impact after 
the contract is signed? On the basis of cur data this seems unlikely ♦ 
Listen to one experienced federal project monitor: 

Once you sign the contract you have very little power to do 
good and substantial power to do evil. You can hassle people, 
but you have a small margin in which to be constructive. 4 
good headstrong contractor can easily tell you where to get 
off* 

We found many instances where the project monitor seemed to avoid clear 
opportunities to influence the conduct of the research* The ethnographic 
component within the DESSI study underwent a major redesign after the con- 
tract was signed. The activity was totally restaffed from what was origin- 
ally proposed, and the decision was made to have a core staff of four field 
workers instead of a larger group of people, one of whom would be located 
at each site.^ However, these changes were not in response to a federal) 
initiative. The major redesign decision on the PI study — to create four 
separate within-program analyses — was influenced by the federal sponsor, 
but on nonmethodological grounds. However, this decision had methodolog- 
ical implications because it reduced the likelihood of extreme formaliza- 
tion and precluded the type of quantitative analysis suggested by the PI 
team in its proposal. This decision was not initiated by the federal 
project monitor. Rather it was an accommodation that the monitor and re- 
search team members made to the fears of a "horse race 11 voiced by officials 
of the four programs being studied. Generally, project monitors took what 
both they and the research team members described as a "collegia!" approach 
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to contract monitoring. In general this meant that the researchers called 
the methodological shoes. 

In contrast to these examples of redesign by contractor initiative are 
two where the federal agency tried unsuccessfully to increase the level of 
formalization. The RES study director faced numerous pressures to increase 
the comparability of the ten case studies (Document C) . All were success- 
fully deflected, first by forcing the preparation of a memo of understand- 
ing which specified that the case studies were to be comparable only ln the 
most general of terms (each was to consider the local Experimental Schools 
project in the context of its school district and the surrounding com- 
munity) and then by insisting that case-study drafts be reviewed by 
experienced ethnographers in the light of that understanding. 

The federal officials monitoring the EBCE contract also sought to turn 
it in a more quantitative direction.; One federal official reports that 
"they really needed to use a larger number of [data] sources and methods," 
but such changes were never made. 

We found only one case of a successful federal effort to affect the 
level of formalization of a project (the CIP study), and here the direction 
of influence was towards less formalization. At the time the contract was 
awarded, the project monitor negotiated an agreement that the project 
director believed required him to hire an ethnographer to work on the 
relevant task, subject to the approval of the federal government. The 
project monitor put the project director in touch with George and Louise 
Spindler, two well-known educational anthropologists. The Spindlers pro- 
vided the contractor with an orientation to traditional ethnography and 
recommended two graduate students who were subsequently hired, but not 
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until they had been interviewed by the project monitor and his independent 
consultants on ethnography. 

Interpretation . In Spite of this rather dramatic counterexample, 
there seem to be severe limits to the client's influence on decisions 
affecting the formalization of research after the contract is signed. 
There are at least three reasons for these limits; changing sponsor priori- 
ties, staff turnover in Washington and contractor expertise. First, in all 
five studies, the commissioning agency lost its initial interest well be- 
fore research was completed. Decisions about the fate of the EBCE and 
Experimental Schools program were made before either of these multi-year 
evaluations could generate research results that could affect program 
policy. In the EBCE case, changing priorities within the funding agency 
clearly gave the project team the opportunity to move in a direction ques- 
tioned by several of its monitors. It was simply not worth the trouble to 
try to discontinue the contract or to take other radical steps needed to 
redirect the work. According to one of its federal monitors: 

[Even] by the time the study began people were beginning to 
lose interest in EBCE. After a year or two [the federal 
agency] was out of the EBCE business. . . and . . . by the end 
of the study programmatic interest was gone. They really 
didn't want to know what had happened. . . . Time had outpaced 
the study. . . . Had [the sponsoring agency] continued to have 
a big stake in EBCE, I could have seen [the team f s] feet be 
really put to the fire. 

Similarly, the arrival of the Reagan administration seemed to limit consid- 
erably federal interest in the results of the DESSI, ?I and CIP studies. 

A second factor is the turnover in federal project monitors. Only the 
DESSI study had the same monitor from start to finish; the RES study had 7 
project monitors over eight years. Generally, the initial project monitors 
had a more active interest in the study than those who came later. 
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Declining interest and project officer turnover might result from the 
length of the research projects. The two studies where declining interest 
were most apparent-RES and EBCE-were also the longest-eight and four 
years respectively. The other three projects lasted 3.5 years (DESSI) , 2.8 
years (PI), md 2.5 years (CIP). Certainly, the length of these first two 
projects made continuity of "federal oversight more difficult. However, it 
should be noted that the big loss of interest in both project's occurred in 
the first two years of each. Moreover, the project with the least turnover 
among project monitors was also rather long (3.5 years). Thus, elapsed 
time alone is insufficient to explain the changed personnel and interest on 
the federal side. 

The third factor, although not as clear, <s that the research teams 
seemed to have an advantage of influence through expertise that, the federal 
project monitors often lacked. Thus, in both RES and EBCE, when agency 
staff had reservations about the work being produced, they commissioned 
special experts or panels to review the results aTd make recommendations. 
However, these groups never made explicit recommendations for clear action, 
such as changing the design of a major study component, that might have 
given the agency justification for exercising its formal authority. All 
five research teams also sought to increase their own claims to expertise 
by turning to eminent scholars and practitioners at crucial points in the 
life of their studies, sometimes at the suggestions of their federal 
sponsors, but more often to challenge them. 

Professional Factors 

At best the client-contractor relationship seems to only generally 
circumscribe the amount of formalization in the design of a multisite 
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qualitative research project. The detailed specification of research 

i 

procedures seems to result more from the predilections of the research 
team, the team's reference groups, and the networks that it uses in seeking 
advice. 

Predilections of the Research Team . The predilictions of the core 
staff of each tean are not always predictable from their methodological 
training. Two of the three people who wrote the RES proposal and provided 
continuity and direction over the life of the project were trained as 
quantitative social scientists. However, the project director — Herriott — 
had just come to Abt Associates from a university professorship. Rather 
than being committed to a particular methodology he was inclined to borrow 
heavily from the most relevant established academic traditions. The es- 
tablished traditions of ethnography and sociological case studies recom- 
mended those approaches strongly to him* Moreover, both he and his deputy 
director for research — Steven Fitzsimmons — were strongly inclined to es- 
tablish the credibility of the project through an emphasis on publication 
in accepted academic journals. 

The people who made up the original team for the PI study, including 
Ward Keesling, Ralph Melaragno, Al Robbins, and Hilda Borko, were also 
trained in quantitative research methods although Allen Smith, an anthro- 
pologist, was added later. However, their experience at SDC also gave them 
reason to welcome alternative methodologies. On the one hand, the prior 
experience of some of the PI team members with the SDC ESAA case study 
project (see Wellisch, MacQueen, Carriere & Duck, 1978) provided a step in 
the direction of more qualitative research. It gave them experience look- 
ing at large amounts of data from a small number of sites, even though the 
data were largely quantitative. On the other hand, other studies in 




the organization— most notably the Title I Sustaining Effects Study—were 
seen by some team members as spending a great deal of time and money to 
learn very little new. At least one member of the PI team noted that 
discontent with classical quantitative studies was "in the air" in the mid- 
1970s. Hence, there was a readiness to move -omewhat away from purely 
quantitative studies, provided that the team could be shown that there was 
a rigorous way to do so. 

The two senior researchers of the DESSI team came together after the 
contract was awarded to The Network. Matthew Miles, a social psychologist 
and educational change expert, was experienced in using both qualitative 
and quantitative methods. He had recently completed a multisite qualita- ' 
tive research project that was not formalized, was dissatisfied with its 
process, and was looking for alternatives (Miles, 1979). One of his first 
tasks as a consultant to- The Network was to critique its plan for 
"ethnographic" research in the DESSI study. His memo drew on his own past 
experience and his knowledge of several other multisite studies that were 
low in formalization. It set the tone for what became the formalization of 
the DESSI qualitative study. His collaborator, Michael Hubersnan, was a 
cognitive psychologist familiar with Piagetian research techniques which he 
describes as "quantifiable even though numbers are rarely used." Their 
shared discontent with less formal approaches and their eclectic methodo- 
logical background seem to have provided the basis for their inventing a 
more formalized approach. 

The.EBCE team had the strongest background in traditional qualitative 
field work. The senior member of the initial team, David Cohen, was an 
intellectual historian more interested in policy issues and theoretical 
development than methodological detail. Although he became j.ess active 
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after the contract was awarded, the three team members, Eleanor Farrar, 
Peter Cowden, and John DeSanctis, had been his students. They obtained 
their field work experience while students largely by on-the-job training 
on projects he had attracted to the Huron Institute. 

Unlike the four other teams, the initial CIP team had no substantial 
interest in qualitative methods prior to being awarded the CIP contract. 
The project director, Kasten Tallmadge, was trained as an experimental 
psychologist; his major assistant, Peter Treadway, was more interested in 
policy issues than any particular methodology. However, one of the 
ethnographers recruited through the initiative of the funding agency, David 
Fetterman, gained the trust of his two colleagues over time. He became an 
accepted member of the core team before the project ended. 

Reference Groups and Advice Networks . Reference groups served to 
reinforce tendencies already alive within the research teams because in- 
dividual members brought their reference groups with them. Thus, when one 
of the long-time SDC employees on PI explained that he had to justify what 
he saw as the relatively unstructured methodology being used to his more 
quantitative peers, the anthropologist countered that he had to explain the 
high amount of structure in the same project to his qualitative colleagues. 
Generally, however, PI team members felt the greatest need to justify them- 
selves to their colleagues at SDC, who were primarily quantitative re- 
searchers. Their colleagues looked for evidence of procedural rigor and 
reliability in what they saw as a deviation from the norms of established 
scientific research. Thus, at SDC the internal reference group was a con- 
servative force that pushed in the direction of increased formalization. 

The DESSI team was also sensitive to a set of reference groups. While 
the study operated through a series of subcontracts and consulting 



agreements that precluded identifying an "office context," the writings of 
senior staff make clear their interest in justifying their work to a larger 
audience concerned with matters of reliability and internal validity as 
defined by quantitative researchers (Document N) . The EBCE team, however, 
presents a different picture. Its members belonged to an "invisible col- 
lege" (Crane, 1972) organized around David Cohen and the Harvard Graduate 
School of Education. It included Jerome Murphy, Milbrey McLaughlin and 
Richard Elmore all of whom were strong advocates for the use of less struc- 
tured research approaches in learning about policy-relevant phenomena. 
Canons of methodological rigor developed by quantitative, researchers were 
much less of a concern to this group. 

Advice networks play an especially important role when the project 
team is making a major departure from its usual mode of operating. Such 
departures were made by the PI and RES teams. The key external advice for 
PI came early on from Ray Rist who was known to the PI team through his in- 
volvement with an earlier study at SDC. His major contribution to the PI 
team was to provide it with a way to use local field workers to collect 
data in a form that would promote cross-site comparability. He showed how 
the combination of extensive initial training on research issues and field 
work methodology, frequent telephone consultation, and the use of open- 
ended instruments could provide adequate structure while allowing for the 
development of a rich description of each site. The legitimacy of this 
approach was later reinforced by another consultant to the PI study, 
Marilyn Gittell, a prominent researcher on community involvement in educa- 
tion. Still, concerns about cross-site comparability led the PI team to 
develop some instruments that were more closed-ended than Rist recommended 
in order to provide protection if his approach did not work. Later, when 
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the team needed continual advice on how to analyze qualitative data, they 
recruited Smith who was trained as a traditional ethnographer but was 
sympathetic to the formalization of qualitative research. He subsequently 
played a major role in implementing a highly formal approach. 

The RES situation was .somewhat different because the task of creating 
useful case studies rested with a cadre of onsite researchers who were not 
recruited until the project was underway. The project team had to find 
individuals who could operate productively under conditions of great 
autonomy, provide them with adequate organizational and intellectual sup- 
port, and create enough confidence in the approach within Abt Associates 
and the funding agency to enable the researchers to function with minimal 
interference for three years, until drafts of their complete case studies 
were available for public review. The latter task was especially difficult 



because of fears that premature disclosure of case study data would 
compromise the ability of the onsite researchers to continue their field 



work (Document C) . Although the on-site researchers at all ten sites were 
continually aware that they could become caught between the competing 



conflict (see Messerschmidt, 1981). 

Rerriott's rural upbringing convinced him that research approaches 
developed in urban settings (such as the sociological commui'iity study) 
would be inappropriate for this project. Re put together an advisory com- 
mittee chaired by Harry Wolcott — an educational ethnographer — that also 
included three other anthropologists and a sociologist committed to highly 
individualistic field work (Howard Becker). Herriott's professional 
contacts were used to assemble this committee. It played an active role in 
selecting the on-site researchers, in reviewing organizational arrangements 



agendas of ,f locals and feds," in only one instance did this lead to serious 
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designed to support them and in reviewing interim case study documents. It 
also provided some legitimation for the overall case study approach and 
reinforcement for the core team vis-a-vis the concerns of Abt Associates' 
nianagement. 

Research Team Recruitment . Taken together, these examples suggest 
that research teams develop a' level of formalization that reflects their 
own predilections and reference groups more than it does the technical or " 
contractual factors presented earlier. If this is true, then a key ques- 
tion is how are. the research teams put together? Basically, they come from 
three sources. The most prominent is a reliance on people already -in the 
"contract shop." Most of thAtaff of every project came from inside the 
contractor organization. (Herriott had an agreement to come to Abt that 
predated the RES proposal effort.) The one exception is DESSI which did 
not have an existing staff because this was to be The Network's first major 
research contract; prior, to thea it was primarily a technical assistance 
organization. However, inhouse availability merely sets the lower limits 
on the pool of available team members. Especially in larger organizations, 
such as Abt Associates and SDC, actual staffing decisions reflect the need 
of corporate managers to make the best allocation of internal staff as much 
as the requirements of specific proposals. Tims, the first PI project 
director at SDC was a person who had spent a number of years as number- two 
man on many projects. He was "due" for a shot at a directorship. Other 
staff were chosen because their existing projects were coming to an end. 
Most of the staff that Abt Associates proposed for the RES contract never 
played their stated roles because the company won another major contract at 
the same time. To accommodate the requirements of both projects, the staff 
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was split, with a number of quantitative psychologists who were less 
interested in an ethnographic approach moving to the other study. 

The second mechanism of recruitment was the use of informal networks. 
These were often formalized through the use of project advisory committees. 
Neither Miles nor Huberman was initially proposed to do the DESSI ethno- 
graphic study; both were to serve in an advisory capacity. When it became 
apparent that the person proposed to do the qualitative study lacked the 
necessary experience with education and contract research and that Miles 
and Huberman had it, David Crandall — President of The Network a.id Director 
of the overall DESSI study — asked^hem to take a new role. Smith came to 
the PI study in a similar manner. Ward Keesling, a senior staff member on 
the PI project, served on an advisory committee for a project on which 
Smith then worked at another organization. Keesling was impressed with 
Smith's work and, when it became apparent that the PI project "needed an 
anthropologist, 11 recommended that he be hired. Where formal advisory 
committees did not provide adequate linkages, invisible colleges often did. 
Thus, Crandall knew Miles and had used him as a consultant before DESSI 
began, and another DESSI advisory committee member known to Crandall — 
Ronald Havelock — had worked with Huberman. Similarly, all the field 
workers of the EBCE study — Eleanor Farrar, John DeSanctis, Richard Elmore, 
and Peter Covden~had been students of David Cohen at various times and 
knew each other well. 

Interestingly, these five project teams were rarely staffed through 
the recruitment of unaffiliated strangers. Although RES, PI, and DESSI all 
made extensive use of systematic recruitment to hire field researchers, 
informal ties played a role. One of the two junior field workers on DESSI 
had worked for Miles before. Four of the ten RES field workers were 
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recommended by members of its advisory committee or by professional 
colleagues of Herriott. However, cone of these individuals became part of 
the central management or cross-site analysis teams on either project. 

There is one other potential influence over recruitment, the federal 
sponsor. All five of these projects had within their contracts a standard 
"key personnel clause" which gives the sponsoring agency the right to 
approve all professional staff assignments. In general this approval was 
routinely granted, both for people proposed initially and those recruited 
later by the contractor. One notable exception, however occurred in the 
case of David Fetterman, the anthropologist who carried out the bulk of the 
ethnographic field work for the CIP team. Since the CIP study was to be a 
replication across four sites of an earlier single-site study which had a 
major ethnographic component, the federal spoosor insisted that the con- 
tractor recruit an experienced ethnographer. When RMC seemed unable to do 
this on its own, the federal project monitor intervened to the extent of 
first putting the project director in touch with the Spindlers at Stanford 
and later interviewing the two candidates whom the Spindlers .recommended. 

Maintaining Autonomy . Once it is assigned or recruited, what does the 
research team do to maintain its autonomy to determine the formalization 
level of a project? There are several resources and tactics that seem to 
be important. The first resource is the general reputation of key members 
of the research team. Miles, for instance, is well known as an expert in 
the planned change field and at, the time of the DESSI study was experienced 
at working on federal contracts and as a consultant to the government. He 
seems to have brought a substantial store of credibility to that project. 
On earlier occasions, the DESSI project monitor had "viewed him in many 
group situations" and always found him "to be a very sensitive and sensible 
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person* 11 Similarly, as noted above SDC had a very strong track record with 
the agency awarding the PI contract which gave them a basis of good will at 
the start of the contract. In addition, according to the project monitor, 
"SDC had in the past been good in knowing when they needed help and in 
bringing in people from the outside. 11 The fact that their proposal men- 
tioned Ray Rist as one of the people they would turn to in getting method- 
ological counsel was seen as a real plus, for he was viewed by the initial 
project monitor as "someone who was being talked about at that time as 
knowing how to do case study research." 

Second, while taking the course they thought technically best, the 
research teams tried to anticipate the concerns of the federal officials 
and thus avoid head-to-head confrontations* One project director explains 
that 

By the time the project officers were brought in, we had beaten 
the issue to death. We had considered all options and knew 
what we were doing so we could make a strong case for what we 
wanted. 

This effort at anticipation characterizes most of the project directors. 

Third, because these qualitative projects were usually embedded ix? 
larger endeavors, it was often possible to use other parts of the projects 
as distractions or buffers. Thus, when one monitor of RES wanted more for- 
malization of its case study component, the project director could address 
his concern through its Organizational Change Study — a primarily quanti- 
tative survey of project implementation in all ten sites that combined 
questionnaire data with centrally structured information provided by onsite 
researchers (see Rosenblum & Louis, 1981). Similarly, the DESSI qualita- 
tive study seems to have been protected bv other parts of the project. The 
centerpiece of this contract was a large survey of 146 school districts. 
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As this survey encountered a number of technical problems, it took up the 
major part of the project monitor's time. Since she had great confidence 
in the professional capability of Miles and Huberman, she devoted less 
attention to their more smoothly functioning part of the contract. 

Fourth, distance and structural arrangements also buffered some 
studies. The 10 RES onsite "researchers were dispersed from New Hampshire 
to Alaska so it was difficult to keep direct tabs on their work. Further, 
the director of RES intentionally minimized contact between the onsite 
researchers and federal officials as a way of maintaining the autonomy of 
the former. Similarly, the subcontracting arrangements on DESSI kept the 
case study team out of direct contact with the project monitor who worked 
primarily through the project's director. Of course buffering could also 
work the other way. A major problem on the CIP project was to convince the 
original team of the legitimacy of ethnography. Federal support for 
Fetterman gave him time to show what he could contribute. 

Finally, project directors can horsetrade and strike deals to maintain 
a desired level of formalization. For example, the RES project director 
arranged to dev :e some of the resources allocated to the book-length case 
studies to a product that would have short-range utility for the agency and 
also demonstrate the competence of the onsite researchers. This product 
contained five chapter-length case studies written by onsite researchers to 
a common format. They were accompanied by cross-case syntheses by five 
well respected researchers and practitioners from different perspectives 
(Document B) . After its publication as a book, this document provided 
visibility for Experimental Schools and its parent agency. 
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DISCUSSION 

The introduction of multisite qualitative research to the policy world 
was part of the methodological eclecticism that characterized that field as 
it expanded rapidly in the 1970s. Although this design had its precursors 
in academic social science, it was largely an invention of federally-funded 
contract research (Herriott 'and Firestone, 1983). Bv the end of the 1970s, 
multisite qualitative studies were a fragile part of the policy research 
scene. There was clearly "something in the air" which made this type of 
study useful to federal research sponsors, but there was great ambiguity on 
the part of bocn sponsors and researchers on matters of ^tudy design and 
implementation. From a historical perspective, the formalization that took 
place in the 1970s was an adaptation to the demands of the policy context. 
Just as quantitative researchers were seeking to enrich their understanding 
by incorporating qualitative elements into their work (see, e.g., Cook and 
Reichardt, 1979) so qualitative researchers borrowed some techniques and 
invented others in order to address canons of good work widely accepted in 
the quantitative world (Smith and Louis, 1982). 

Yet, the mix of short-term forces that influenced the amount of for- 
malization in these five projects is somewhat surprising. The picture that 
comes through is not of a systematic effort to change the character of 
qualitative research to fit its new setting. Indeed, there seems to have 
been disagreements among both researchers and research sponsors on the im- 
portance of the then widely accepted quantitative standards of excellence 
(Rist, 1977). Instead, each project was designed through a process that 
most resembles a "garbage can model" of decision making (see Cohen, March, 
and Olsen, 1972). From this perspective research design decisions are not 
the product of rational deductions from explicit policy questions or 




methodological cations (Martin, 1982). Rather they are the result of 
processes whereby decision makers come together under pressures of time and 
competing commitments and where their decisions depend primarily on the 
preconceived solutions and problems they bring to the decision setting. 

Such appears to be the situation with respect to the five projects we 
studied. In general the requests for proposals (RPPs) we reviewed reveal a 
need to accommodate a wide range of political preferences and design pos- 
sibilities within the funding agency without acknowledging either inherent 
incompatibilities or the fact that not everything called for could be 
accomplished at the suggested budget level. The proposals prepared in re- 
sponse to these RFPs r £ 7eal a reluctance of competitors to commit them- 
selves to an explicit course of action, partially because of the ambiguity 
of the RFP but also for fear of alienating those who would be judging their 
capabilities to carry out t'e research. (Thus the need for the "intricate 
dance" referred to above.) 

What the federal sponsor generally buys is not an explicit research 
desisn, but a general statement of organizational capability (current 
staff, preferred consultants, past work, etc). Only afte r a contract has 
been awarded, when the full range of considerations can be evaluated with 
relatively greater impunity, can concrete design decisions begin to be 
made. At this point the contractor's team gains the upper hand, first be- 
cause it typically outweighs the federal sponsor's representatives in 
number, effort level, and technical expertise, and later because it has 
greater access to knowledge about the phenomenon under study and can argue 
that the sponsor's original assumptions about the phenomenon are no longer 



valid. 
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Still, the contractor's team often faces many difficult questions 
about how best to proceed. The definition and resolution of these issues 
is also subject to garbage-can forces that affect project staffing as well 
as internal activities. All the teams we studied were assembled through 
chaotic processes that resulted from the need to propose a credible staff 
on short notice while taking into account the existing commitments of cur- 
rent staff members. The network of weak ties between current* staff and the 
larger world of policy researchers also affected the way recruitment deci- 
sions were made. In no case was the full team that eventually designed and 
implemented these five studies identified before the contract was awarded. 
Further, regardless of how or when a team was recruited, its members 
brought with them a variety of generic preferences about how to conduct 
research. When the proposal was being written these personal preferences 
began to affect the study's design, but they were largely subordinated to 
the shared goal of winning the contract. However, once that hurdle was 
overcome, both the general and specific elements of study design were sub- 
ject to redefinition through negotiations within the research team. 

, In two respects, this garbage-can process is "messier" than one might 
like to tfelieve the research design process should be. First, technical 
design considerations such as research purposes and the number of research 
sites have less impact on how the study is carried out than they would if 
there were widely accepted guidelines for how to carry out multisitr 
qualitative studies. Second, the balance of influence between the federal 
sponsor and the contracting team is tilted more towards the latter group 
than ought to be the case according to the normative depictions of Baker 
(1975) and Coleman (1972). 
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Such messiness may be more healthy than pathological, however. It 
steins from three characteristics of qualitative contract research. First, 
qualitative research is inherently a "nonroutine technology" (see Perrow, 
1967). Such a technology is particularly useful in situations where the 
rules for proceeding are unclear and there are many ' unanticipated and dif- 
ficult to analyze events. Hence, the importance of "the researcher as 
instrument" (Sanday, 1979) and the difficulty of deriving a course of 
action primarily from technical considerations. 

Second, multisite qualitative research is a relatively new enterprise 
that still requires a great deal of trial-and-error learning to be done 
well, (hie of our informants, a person formerly responsible for top-level 
research planning within the National Institute of Education, was highly 
understanding of the type of design inconsistencies that we found and 
skeptical of any attempt to develop a "textbook approach" to qualitative 
policy research. "How can one hope to produce a Campbell and Stanley 
(1966) [for multisite qualitative research]," he asked, "when the field has 
yet to discover its Fisher?" (Smith, personal communication, 1981). Al- 
though we observed much commitment on the part of the teams we studied to 
codifying, justifying, and communicating their craft (see especially 
Appendix D, Documents C, I, L & N) , they have to date touched upon only a 
few of the many Issues that need to be addressed in a comprehensive over- 
view of the full range of design and implementation choices in multisite 
qualitative studies. 

Finally, the organizational arrangements for conducting policy ; re- 
search are inherently unstable. At their core is an interorganizational 
arrangement, quite similar to the intergovernmental arrangements that were 
used for many of the social action programs of the 1960s and 1970s. There 
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is no reason to believe that such arrangements should provide substantially 
more federal control over the research implementation process than they did 
over the policy implementation process (Bardach, 1977). Moreover, each 
research project is a temporary system with different rhythms of events 
within its different components. As the federal project monitor respon- 
sible for this study put it, "contractor time is different from academic 
time, and both of those are different from the time of federal officials. ,f 
As a result interest waxes and wanes at different times for federal offi- 
cials and the research team, and the cast of characters — especially on the 
federal side — changes frequently and often dramatically. The greater sta- 
bility and continuing interest of the contracting team tends to give it 
more influence over design decisions. 

How do these observations to help assess the utility of highly for- 
malized multisite qualitative studies? In some ways such an assessment is 
premature 'because this approach is still so new. We have seen useful 
research conducted at all three levels of formalization that we observed. 
Nevertheless, formalization, at least to a point, seems to have distinct 
advantages. The development of an initial conceptual framework and its 
operationalization through a series of open-ended instruments is extremely 
useful for ensuring comparability in data collection across sites and re- 
sponsiveness to the original research issues identified by the client. 

The more formal data reduction and analysis techniques also facilitate 
drawing conclusions. They provide a much more precise language through 
which members of a research team and reviewers from the sites studied can 
describe and debate conclusions about specific settings and then about 
cross-site patterns. This language forces the team to confront differences 
of perception so that conclusions can be "audited" (Lincoln and Guba, 
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1982) , and the agreement of a group of well-informed experts becomes a 
major claim for the credibility of findings. 

Whether these techniques constitute a major advance in the reliability 
and validity of qualitative research is more open to question. By them- 
selves, they cannot constitute stronger "proof" for the uninformed reader. 
A great deal of researcher judgment goes into the development of the type 
of ratings utilized by both the PI and the DESSI teams-much more than goes 
into the numbers analyzed in survey or experimental studies. The reader 
must take it on faith that these judgements are correct. Typically, such 
judgements are less well justified in the final report of a highly formal- 
ized study than in that of research using a more traditional ethnographic 
approach where substantial excerpts from original field notes are shared 
with the reader. The authors of some formalized studies point out fhat 
case study materials are available for external audit, but these are gen- 
erally difficult to use by individuals who did not do the original field 
work. In sum, techniques of formalization in multisite qualitative studies 
have advantages and disadvantages as means to bolster the credibility and 
utility of a research report. Their wider use will depend in part on time 
and cost implications. Because they are fairly expensive to employ, we 
venture the prediction that they will become an important part of the "tool 
kit" of multisite qualitative researchers without becoming the sine qua non 
of good practice. 

The issue for those who commission and conduct qualitative policy re- 
search seems to be one of deciding how much formalization is appropriate 
under particular conditions. What is needed is not a textbook of the 
"best" way to do multisite qualitative research but a contingency approach 
to study design and implementation. As the field matures, such an approach 
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ought to give progressively more weight to the political and scientific 
imperatives generated by the policy issue itself (Cronbach, 1982) and less 
to the personal and professional priorities of the research team. However, 
this objective may be somewhat idealistic for both qualitative and quanti- 
tative research. If our observations about the world of policy research 
apply beyond the five projects we studied, further formalization of quali- 
tative policy research will not be achieved quickly or without controversy. 
Hopefully, the experiences of the past decade have helped us to better 
understand its major elements. 
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NOTES 



This paper has been prepared with support from the National Institute 
of Education under contract No. 400-80-0019. It does not, however, 
necessarily reflect the view of that agency. We are particularly 
indebted to Fritz Mulhauser of the Institute's staff for his unfailing 
facilitation of our research. 

We are indebted to Miles (1979) for pointing us in this direction. 
Although he speaks of the "bureaucratization" of field work and 
Talmadge & Rasher (1981) refer to the "quantification" of qualitative 
data, we have chosen the term "formalization" to reflect a phenomenon 
broader than either data collection arrangements or data reduction 
techniques. 

No effort was made to achieve a random sample of projects within each 
of the five relevant cells of Table I. Instead we endeavored to 
select a sample representative of the field qualitative policy re- 
search in the 1970s by emphasizing variation on the following seven 
factors: the funding agency, the contractor organization, the date of 
contract award, the size of the contract, the length of the funding 
period, the previous experience of key federal monitors and the dis- 
ciplinary background of key project staff. We also gave priority to 
projects that our informants in the snowball sampling process sug- 
gested were methodologically sophisticated. For comparable data on 
all 25 projects, see Appendix A. 

The on-site researchers included Allan F. Burns, Charles A. Clinton, 
A, Michael Colfer, Carol J. p ier ce Colfer, William L. Donnelly, Ronald 
P. Estes, Jr., William A. Firestone, Lawrence Hennigh, Stephen J. 

8<*7 



Langdon, Donald A* Messerschmidt , Marilyn C. Richen, Charles I. 
Stannard and C. Thompson Wacaster. In addition to their case study 
reports these anthropologists and sociologists produced a lively lit- 
erature on the stresses and strains of qualitative field work in the 
policy research setting — see Appendix D, Document A, for illustrative 
citations . 

In those instances where the authority for our characterization is a 
public document, that document has been cited here and annotated in 
Appendix D. In all other instances we have relied on our interview 
notes . 
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APPENDIX A. PROFILES OF 25 MULTISITE QUALITATIVE STUDIES 



Information about the organization/ design and 
implementation of 25 multisite qualitative policy research 
studies can be found in this appendix. Table A-l presents a 
brief overview of the project within which each study was 
located, it is followed by a detailed "profile" for each 
project. Section I of each profile describes the total project. 
For those projects without distinct substudies (see item 12)' 
Section II also presents details about the total project. 
However, for projects with distinct substudios, Section II 
describes only a single substudy. in general this substudy is 
the "most qualitative" one in its approach to data 
collection/reduction/analysis/reporting. 

Explanatory notes for each generic item in the profile can 
be found in Appendix B. 
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Table A-l. Roster of 25 Federally-funded Policy Research Projects 
Having at Least One -Primarily Qualitative Component • 



ID 



Project Title 



Funding 
Agency 



Contractor 



Personnel 



01 The Evolution of Performance 
Contracting in Five School 
Districts 

02 Pilot State Dissemination 
Program Study 

03 The Rural Experimental 
Schools Study 



04 Federal Programs Suporting 
Educational Change 

05 National Evaluation of 
Project Developmental 
Continuity 

06 Title I Allocation 
Demonstration Study 

07 Documentation and Technical 
Assistance in Urban Schools 

08 Compensatory Education Study: 
Administration of Title I 

09 Case Studies in Science 
Education 

10 Evaluation of Project Inform- 
ation Package Dissemination 
and Implementation 

11 A Study of Experienced-based 
Career Education 

12 Program Consolidation and the 
State Role in ESEA Title IV 

13 Vocational Education Equity 
Study 

14' A Study of the R&D Utiliz- 
ation Program 



HEW/ASPE Rand 



OE/NCEC 



OE/ES 
NIE/ES 



OE/OPBE 
HEW/ACYF 

NIE 

NIE/SCPS 
NIE 

NSF/DSE 
OE/OPBE 

NIE/E&W 
OE/OPBE 
OE/OPBE 
NIE/DIP 

w 4 



Columbia/ 
BASR 

Abt Assoc 



Rand 



High 
Scope 



Abt Assoc 



Ctr for 
New Schls 

Booz- 
Allen 

Univ of 
Illinois 

AIR/RMC 



Huron 



Rand 



AIR 



Hall 

Carpenter 



Sieber 
Louis 

Herriott 

Fitzsimmons 

Kane 

Berman 
McLaughlin 

Love 

Powell 

Bond 

Vanecko 
Ames 

Wilson 
Runkel 

Beaven 
Goettel 

Stake 
Easley 

Campeau 
Binkley 



Farrar 
DeSancti > 

McLaughlin 
McDonnell 

Harrison 



Abt Assoc Louis 
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Table A-l. (Continued) 



ID 



Project Title 



Funding Contractor Key 
Agency Personnel 



15 Longitudinal Implementation 
Study of PL 94-142 

16 The Career intern Program 
Study 

17 The Teacher Corps 
Evaluation Study 

18 The Youthwork National 
Policy Study 

19 A Study of Dissemination 
Efforts Supporting 
School Improvement 

20 A Study of Parental 
Involvement in Federal 
Programs 

21 District Use of Information 



22 



23 



24 



25 



OE/BEH 

NIE/T&L 

0E/0ED 

DOL/OYP 

0E/0ED 

0E/0ED 



NIE/T&L 
0E/0ED 



Case Studies of Interorganiz- Nie/DIP 
ational Arrangements for 
Knowledge Utilization: II 

Case Studies of Interorganiz- NIE/DIP 
ational Arrangements for 
Knowledge Utilization: III 



SRI 



RMC 



SRI 



Cornell 
Univ 

The 

Network 



SDC 



Huron 



Case Studies of Interorganiz- NIE/DIP Abt Assoc 
ational Arrangements for 
Knowledge Utilization: I 



A Descriptive Study of 
Significant Bilingual 
Instructional Features 



NIE 



American 
Univ 



TDR 



Far West 
Lab 



Stearns 



Tallmadge 
Fetterman 

Marciano 
Deslonde 

Rist 



Crandall 
Huberman 
Miles 

Melaragno 

Keesling 

Smith 

Kennedy 



Yin 

Qwaltney 



Havelock 
Huberman 



Chin 
Herzog 

Tikunof f 



Note: 



The 25 projects are in ID sequence from the earliest funded 
to the latest. See the following page for a glossary of 
funding agency abbreviations. 
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Table A-l. (Continued) 



GLOSSARY OF FUNDING AGENCY ABBREVIATIONS 



DOL/OYP Department of Labor/Office of Youth Policy 



HEW/ACYF Department of Health, Education & Welfare/ 
Agency for Children, Youth and Families 

HEW/ASPE Department of Health, Education & Welfare/ 

Assistant Secretary for Planning & Evaluation 



NIE National Institute of Education 

NIE/DIP National Institute of Education/ 

Dissemination and Improvement of Practice 

NIE/ES National Institute of Education/ 

Experimental Schools Program 

NIE/E&W National Institute of Education/ 
Education and Work 

NIE/SCPS National Institute of Education/ 

School Capacity for Problem Solving 

NIE/T&L National Institute of Education/ 
Teaching and Learning 



NSF/DSE National Science Foundation/ 
Division of Science Education 



OE/BEH Office of Education/ 

Bureau of Education for the Handicapped 

OE/ES Office of Education/ 

Experimental Schools Program 

OE/NCEC Office of Education/ 

National Center for Educational Communication 

OE/OED Office of Education/ 

Office of Evaluation and Dissemination 

OE/OPBE Office of Education/ 

Office of Planning, Budgeting and Evaluation 
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BEST COPY AVAILABLE 

PROJECT PROFILE 

Section I 

3. Funding Agency BEH/ASt& (Contract Mb. ^-03-70-1^) — 

4. Project Monitor aTjjagj Gk^n , Telephone (202) 2h«U*fii 

b * Telephone 

5. Research The Rand Ccortwaticn c c*»— t " 7 

O' Duration z»7 years 
3Mt,lhto ' a 901,06 Total Badge s 350 K 

10. Project Director a. foerge Hall Telephone " 

b. Margaret Surpentar-guffnan Telephon ed) jj^gg 
tl. Overall Research Design a. ^qualitative studydes) only 

b - — separate qualitative and quantitative substudles 
c - — .integrated qualitative and quantitative substudy(les) 
12. Name of Distinct Substudles 13. Substudy 14. Principal Investigator 15. Appx 

Tm Budget 
a.aerlaw of Parf. CoatatGtinir Ql Stuelear/Rall c ifn r 



e. 



5S£tlon H (Cm. studiaa of Performance Contracting) 

16. Unit of qualitative Study School Putrid 17 . No. 0 f Units (C) g 

18. *des of Field Research (C) 19. Structure of Field ResearcTTcT 

5 - — vJslS Zl — iS P8r V 3 ? !5 a. _2_ one person throughout 

c' -=-7nT 8 ™ilSn?Jn^- Per v1s1t b ' -a. «r1es of sole persons 

a' -r- l"!!!™" 8 "! 1 ^ over "»s. c. ~F team of 2"Tersons 

d. continuous for _ months — d. ± team of ^ 

20. Techniques of qualitative Dita Collection (R) 21. Number of Field Workers (C) 

i: ± asaarsss^ »• ^ r o; is F?ii p j 1 ;:rU d ^ flca - 

c - highly-structured Interviewing 6ld W ° rkers {C) 

d. X: semi-structured Interviewing a . - anthropology 

largely-unstructured Interviewing b . T~ education 

f. highly-structured observation c . - political selanca 

g. ^. semi-structured observation 5 ~ MycholL 
f • JL. largely-unstructured observation l. "f" Jocloloo? 

1. _g_data acquisition from scfonl rii ft « f. ~ other 

23. Types of Public Unit-specific Narratives (C) a. none " !" 

b « -JL. brief vignettes 

c - -X chapter-length case studies 

«• «js. book-length case studies 

24. Facts of Publication of Most Informative Study Oocument(s)- 

** ^"t-r, Polly »nd Oeorga R, Hall. Case Studies la "sducational Performanca 
v ggagBgL* 5£i i*> Conclnaiona ag^plications (Wand Report ^bb/l HS v). 
Santa Monica, CAT The Hand c\xrporaHo^,^7i., 51 pp. 

b. Carpaatar-Sjf fman, Ifergarat, Georga R, Hall and Gerald C. Sunnar. Chan K9 in 
^^^ s 1? |g2^hite from Performance; Contractl^. Cambridge, As 



Section £ 

1. ID 02 

3 



PROJECT PROFILE 



2. Project Name Pilot State DiaseTnl nation Prcgran Study 



Funding Agency OE/NCEC (Contract No« 05C-0-70493Q) 

Project Monitor a. John Coulson Telephone (202) Z$\x-Sh70 

b. Telephone 

Research Bureau of Applied Social aaaearch 6. Start 9/70 7. £ n d 6 / 72 
Organization c^umbia UaiT^aity 3# (juration " ' ' 



Hew Tork, HT 10025 



10. Project Director a. Sam D» Siaber 

b. Karen Seashore Lonis 

11, 



years 

9. Total Budget $ ^° s 
Tel ephone 



Tel ephone (617) 5*42 -7037 



12. 



Overall Research Design a. qualitative study(les) only 

b* separate qualitative and quantitative substudies 

c * x integrated qualitative and quantitative substudy(les) 

Name of Distinct Substudies 13. Substudy U. Principal Investigator 

Type 



a. State Stady 

b. Linker Study 



c. User Stady 

d. ^ 
e. 



Ql/Q n 
Qn 



HA. 



TIT 



15. Appx. 
Budget 

$ 1?0 K 
$ T55T 

s 

$ 



Section H^: Linker Stu^jr 

16. Unit of Qualitative Study School District 
18. Modes of Field Research (C) 



a. visits for days per visit 

b. visits for days per visit 

c. 12 intermittent for J[£ over 11,05 • 

d. - continuous for months 



17. Ho. of Units (C) 12 

19. Structure of Field Research (C) 
a. 2 one person throughout 



20. 



_ sole persons 
persons 
persons 

Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) 



b. - series of 

c. 1 team of 

d. • team of 



a. document acquisition 

b. ij questionnaire administration 

c. - hi ghly- structured interviewing 

d. X semi -structured interviewing 

e. 2 large! y-unstructured Interviewing 

f. highly-structured obsarvation 

g. 3 semi -structured observation 

h. mmMm largely-unstructured observation 
1. 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

* a. 2 anthropology 

b. 2 education 

c. political science 

d. 1 psychology 

e. sociology 
JL, rt ht 



23. 



24. 



Types of Public Unit-specific Narratives (C) a. none 

b. - brief vignettes 

c 3£2 chapter-length case studies 

d. book-length case studies 

Facts of Publication of Most Informative Study Document(s); 

Louis, Karen Seashore and San D. Sieber. Bureaucra cy and the Dispersed Organization , 
Norwood, MJ: Ablax, 1979. "* ' " s 
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Section I_ 
10 03 



PROJECT PROFILE 



I. 
3. 
4. 



2. Project Name Iha Sural Sgariagtal Schools Study 



Funding Agency OB/SS thanllE/ES (Contract Koa. 030-0-72-5215; U0O-76--OQU)" 



Project Monitor a. _Dgrld Budding 
b. John Bfteraeier 



5. Research Abt Aaaogjataa Inc. 



££t) Telephone (617) aS>2-7100 
(7th ) Telephone (202) 2&-6050 



10. 



11. 



Organization Z TT" ' 

55 Wheeler Street 

Canfaridgo, Ml 02138 

Project 01 rector a._Robart B. Harriott 



6. Start _iy_72 7. End __6/j0_ 
,8. Duration 8.0 years 
9. Total Budget $ 5*1 H 



Tel ephon e (617) 369-9779 
Tel ephone 



Overall Research Design a. ^.qualitative study(ies) only 

J- -£_separat« qualitative and quantitative substudles 
c. .integrated qualitative and quantitative substudy(les) 



12. Name of Distinct Substudles 13. 



Pupil Change Study 



Substudy 14. Principal Investigator 
Type 



c. Organizational Change St udy 

d. Community Chango Statfr 

e. Special St»r|i flg 

Section II: 



01 
Qn/Ql 

OA .. 



16. 



Site Case Studies 
.Unit of Qualitative Study 



Stephen J t Fltzsimnong 
Wendy Peter Abt 

Shu-n* gpgenblnn 

Stephen J t htzap man* 
Robert E. Harriott, 



15. Appx. 
Budget 

1 4& « 

$ 1.2 M 
$ 0.8 K 
$ 0.5 M 



School District 



18. Modes of Field Research (C) 



19. 



17. No. of Units (C) 10 



20. 



23. 



24. 



*• visits for days per visit 

°* —Z. — visits for days per visit 

c. __l_ intermittent foroOO davs over 30 mos. 

d. _2. continuous for _J0 months (full-time) 
Techniques of qualitative Data Collection (R) 21 

a « -JL document acquisition 

b. questionnaire administration 

c. -highly-structured interviewing 

d. h semi -structured Interviewing 
-J. largely-unstructured interviewing 

f. "1ghly-structured observation 

g. - semi -structured observation 

h. 1 largely-unstructured observation 
1 • -£.-3tiH photography 



Types of Public Un1t-spec1f1c Narratives (C) a. 

b! 



Structure of Field Research (C) 

a * JL. one person throughout 
b - 4_ sarles of J2_ sole persons 
c 1 team of _2__ persons 
d. . team of persons 

Number of Field Workers (C) I 

22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

a - 8 anthropology 
Q * JL. education 
c jz~ Political science 

d. - psychology 

e. «2_ sociology 

f • 



nont 

10 brief vignettes 
SI chapter-length case studies 
JL book-length case studies 
Facts of Publication of Most Informative Study Oocument(s) : 

* # ^SS^^^L? 5 ^ I ^ tl ™ "l^Si School togagmgnt , Flndl^ 3 

S!^!!?* 1 * R ?°« rt B » "Tensions la Research Design and Weaentakon- «Ph* 
fcqperiaental School* Study,- American Behavioral KwSTuSa 26 (?) ^S) 
c Harriott, Robert B. and ^areeTTBi^ 

Chauge, Case Studies, and Analyses. Ber^eyT^TS^I,^^^^ 

• * 
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PROJECT PROFILE 

Section £ 

^* m Oh 2, Project Name Federal Pro grans Supporting Educational Change 

3, Funding Agency OE/OPHS (Contract Uo. ^ ) 

4. Project Monitor a. . Telephone 



b. Anne Bezdak (Vfeinholiagr) (2nd ) Telephone (202) 2iig^877 

Research The Hand Camoration . *± 6. Start 6 /73 7, End 14/77 

Organization 170Q ^ str0fft 8< 0urat1on 3<s > years 

Santa Monica, & 90I4O6 0 9, Total Budget $ 1*2 H 



10. Project Director a . John Pincna (1st) Telephone (213) 393-Qlll 

b. Paul Barman (2nd) Tel ephone_ 



11. Overall Research Design a* qualitative $£udy(1es) only 

b * x separate qualitative and quantitative substudles (Phase I) 
c. x integrated qualitative and quantitative sucstudy(ies)(Phaser 

12. Name of Distinct Substudies 13- Substudy 14. Principal Investigator 15. Appx. 

Type 3udget 

a. Phase I National Surrey Qn NA $ 

b. Phase Z Field Sanoarch Ql $ 

c. Phase II National Surrer On NA_ $ 

d. Phase II Field Resq?rt»h QI HA $ 

*• $ 

Section m Phase II Field Research (ESEA Title HI St VII Projects) 

16. Unit of Qualitative Study School District 17, No. of Units (C) 30 

13. Modes of Field Research (C) 19. Structure of Field Research (C) 

a - 30_ JL_ visits for jj_ days per visit a. one person throughout 

_jz visits for days per visit b. series of sole persons 

c. intermittent for days over mos. c. 22L team of 2_ persons 

d. continuous for months d. mm - m team of persons 

20. Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) 2£ 

a - JL document acquisition 22. Major Disciplinary Identl flea- 

fa. questionnaire administration tlon of Field Workers (C) * 

c. highly-structured interviewing 

d. semi -structured interviewing a. ~ anthropology 

e. largely-unstructured interviewing b. _~ it education 

f - ^highly-structured observation * c 4 _7_i)ol itical science 

g. j>_ semi -structured observation d. m J Lm psychology 

h. m largely-unstructured observation e* - sociology 

1. f . 

23. Types of Public Unlt-spec1f1c Narratives (C) a. x none 

b- — _ brief vignettes 

c. chapter-length case studies 

do book-length case studies 

24. Facts of Publication of Most Informative Study Oocument(s) : 




* 2Iajor enphasis upon the use of v ell-trained and experienced social scientists as 
field workers* 
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Section I 



PROJECT PROFILE 



1 . 
3. 

5. 



ID 05 



2. Project' Name National Evaluation of Pro.jact IWo™.^ Coottooity' 
ncv HEW /Arm? /r^.^. »t rw* - A — ■ * 



Funding Agency HEW/ACTF (Contort Nos. HEfr 100.7^33. J iq^T^ 



Project Monitor a. Sathar Kraah 

b. 

XS???** High/Scop. Ed. R. S . Faind.t^' 
Organization IS) y. m w strart 

Tpallanti, MI U8L97 



Telephone (202) u72.3g9h 
Telephone 



10. 



11 



12. 



Project Director a. John Log 



6. Start n /7U 7. End Jj 82_ 

8. Duration J»5 years 

9. Total Budget $ 2.0 M 



b.Juis I, Bond 



Tel ephon e (203) 227-7201 



i^rt) Telephon e (313) u85-200O 



Overall Research Oeslgn a. ^qualitative study(les) only 

b. jcseparate qualitative and quantitative substudles 

c * — integrated qualitative and' quantitative substudy(les) 

Name of Distinct Substudles 13. Substudy H. Principal Investigator 15. Appx. 



& ami, sta,^ 

0. Impact S^rfr 

c. 
d. 

e. " " 



Qn/Ql 



Lav*. Snnh. 



0ranrlll« t Bond. 
Vackat* 



Section lit Iapact Stady 

16. Unit of Qualitative Stud y PDC Sif 
18. 



Budget 

j Ui.ll 

$ 
$ 



17. No. of Units (C) 11 



20. 



Modes of Field Research (C) 

»• v1 s1ts ftr days per visit 

b - -^.—.visits for days per visit 

c. intermittent fop days over mos. 

<*♦ continuous for months 

Techniques of Qualitative Data Collection (R) a. Number of Fielders (C) 16 



19. Structure of Field Research (C) 
a. 6 one person throughout 
b - -I- series of sole persons 

c - " team of persons 

»• team of persons 



a. document acquisition 
o«-__-l. questionnaire administration 
*• -f- h Wy-structured Interviewing 
d. _1_ semi -structured Interviewing 
-1- J*rgely-unstructured Interviewing 
highly-structured observation 

g. senrf -structured observation 

h. ^rgely-uns^ructured observation 



1. 



23.. 



24. 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

a. anthropology 

b. _2_ education 
C - political science 

psychology 
e. 2 sociology 
f. 



Types of Public Unit-specific Narratives (C) a. __x. none ~~ 

b. brief vignettes 

c« chapter-length case studies 

' d * book-length case studies 

Facts of Publication of Most Informative Study Documents )• 



9 
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PROJECT PROFILE 



Section I 



1 . ID 06 2. Project Name Title I Allocation Demonstration Study 



3. 


Funding Agency ins/Titla I Study Group 


(Contract No. bCO-75- ) 




Project Monitor a •Anna Milne 


Telephone 




b. 


Telephone 


5. 


Research Abt Associates Inc* 


6. Start 1 / # 7. End 12/ 78 




Organization 55 Ynjeeler Street 


8. Duration b.O years 




Cambridge, MA 02138 


9. Total Budget S 3.8 H 


10. 


Project Director a. Janes Vanacko 


(1st) Telephone (617) 661-6508 




5. Nancy Anas 


(2nd) Telephone (617) "92-7100 



11, 



12. 



Overall Research Design a. qualitative study(ies) only 

b. separate Qualitative and quantitative substudies 

c. x i ntegrated qualitative and quantitative substudy(ies) 



Name of Distinct Substudies 13. Substudy 14. Principal Investigator 

Type 



15. Appx. 
Budget 



a. Student-level .Allocation 


On 


NA 


S 130CK 


b. Parent Surveys 


On 


NA. 


S 900TC 


c. Innlementation Study 


01 /Cn 


Catherine Baltsell 


S 70CK 


i. Cost Studr 


Oil 






e. 








Section II : lanLamarrtatlon Studr 








16. Unit of Qualitative Study School District 


17. No. of 


Units (C) 13 



18. Modes of Field Research (C) 



19. Structure of Field Research (C) 



*• -12. -Jl visits for days per visit 

b. visits for days per visit 

c. intermittent for days over roos 

d. T continuous for months 



20. 



a. = Qne Person throughout 

b. series of sole persons 

c. n team of 2 persons 

d. - team of persons 

Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) Jtf 

a. h document acquisition 

b. - questionnaire administration 

c. T~ highly-structured interviewing 

d. T~ semi -structured interviewing 

e. ; - 1 argel y^unstructured interviewing 

f. highly-structured observation 
9. J_ secrl -structured observation 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 



• largely-unstructured observation 



a. - i anthropology 

b. — education 

c. political science 

d. JL. psychology 

e. JLm sociology 

f. 5 other 



23. 



24. 



Types of Public Unit-specific Narratives (C) a. — _ none 

b. 23^ brief vignettes 

c. chapter-length case studies 

d. book-length case studies 

Facts of Publication of Most Informative Study Document(s): 

7anecko> Janes and Nancy Ames (with Francis Archambauit), '-'ho Senefits from 
Federa l Education Dollars? Title I Allocation Policy ♦ (Snhridgej i^A: 
Abt Books, 1919, 28o pp« ~ 



( J5 
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PROJECT PROFILE 



Section I 



1. 
3, 
4. 



10 -21— 2 ' ***** Naa« Documentation and Tachni^ T. Assists fa g^n <^w, g 
funding Agency MIS/SCP3 (C^temct Ho, MIS-75- } ™ n l>rnnP ^ 



Project Monitor a . Frederick Mhlhauaer 
b. 

Research 
Organization 



Center for Nat) Schools 
h31 South Dearborn St, 
Chicago, 3L 6060$ 



Telephone (202 ) ______ 

Telephone 



10. 



12. 



Project Director a., 
b. 



Thonaa A. Wilaon 



. 6. Start ___/___ 7. End ____/__q_ 
,8. Duration $.£ years 
,9. Total 8udget $_______ 



Tel ephon e (312) 939-7025 
Tel ephone 



Overall Research Design a. r "qualitative study(les) only 

°* separata qualitative and quantitative substudles 

c « Integrated qualitative and quantitative substudy(ias) 

Name of Oistinct Substudles 13. Substudy 14. Principal Investigator 15. Appx. 



a. Cagg Studies of _______ 

b. Caae Studies of TA Taara 

c. De yelotnent of Demo. Mod el 

d. 3r 
e. 



Type 

Ql 



01 

3£ 



Stave W31so n/PhiI R___»l 
Don Moore/D ick Schmnclf 
Ton Wilson 



Budget 

S . 1-8 > ; 

S 1-2 M 



Section _i : Caae Stadia* of Projects 
1 6. Uni t of qual 1 tatrl ve- Stud y School Change Project 
18. Modes of Field Research (C) 19 
a. - visits for _days per visit 



17. Mo. of Units (C) 9 



Structure of Field Research (C) 



20. 



b. visits for days per visit 

c « 3 Intermittent for days over mos 

d. ___ continuous for months (full^e) - _-_ tC4B1 ur per5cns 

Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) H 

j-- ^ . _ . ^ . — 



a. ___ one person throughout 

°« __. series of sole persons 

c. _ team of _____ persons 

d. team of perscns 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

a. 3 anthropology 

b. _____ education 
c « -TL. political science 
d - Jl. psychology 
e. __. sociology 



23. 



24. 



a. _J document acquisition 

b. ___ questionnaire administration 
c - highly-structured Interviewing 
d. _6_ seal-structured Interviewing 
«• ___ largely-unstructured interviewing 

f. - highly-structured observation 

g. ____ semi -structured observation 

h. _1_ largely-unstructured observation 

1 • 2 T?hnft, ' f "^Tihr ! 

Types of Public Un1t-sp*c1f1c Narratives (C) a. none 

b. __[ brief vignettes 

c. «• chapter-length case studies 

d. _£ book-length case studies 
Facts of Publication of Most Informative Study Dccument(s): 

^ 'ihS^Irt ""S* J' ^ S*?* *5 Action : A Case Study of 



b. 



^£^^^^^53? Tte, CTA Experience , New Xoric: 
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PROJECT PROFILE 



Section 1 
ID 08 



1. 
3. 
4. 



2. Project Name Conpensatoir Education Studr: Administration of Title I 



Funding Agency NIE/Tltla I Study Group (Contract »o« NI£-76-CO$7) 

Project Monitor a. Donald w. Barms Telephone (202) 2fli-607O) 

b * Telephone 

Research 



3o 07 '■All an & Hacilton* Inc« 

Organization 7ZZ r T! I ! ;T7r 
1025 Connect lent Avq», N«W» 



10. 



12. 



Washington, D+C;- 20036 



Project Director a. Douglas Beaven 
b. 



6. Start 2 / 76 7. End g / 77 

8. Duration 0«8 years 

9. Total Budget $ 8?0 S 

Tel ephone (617)1j234j3CO 



Tel ephone 



Overall Research Oesign a. qualitative study(ies) only 

b * JE_ se P arat s qualitative and quantitative substudles 

c - integrated qualitative and quantitative sub*tudy( ies) 

Name of Distinct Substudles 13. Substudy H. Principal Investigator 

Type 

a. Stata Surrey (fl-$0) Qn 

b. State Case Stadias Tn^B) Ql 

c. ~ 

d. ~ — — 

e. — — - 



Douglas Heaven 
Robert Goettal 



15, Appx. 
Budget 

S 350 S 
$ 500 K 
$" 



Section IT: Stat* Case Studies (conducted as a subcontract to Syracuse UrrLv* Res* Corp.) 
16. Unit of Qualitative Study Stata, Education Agency 17. No. of Units (C) 8* 
18. .Modes of Field Research (C) ~ 



a - -8 — visits for JL_ days per visit 
b * ZL-JL v1s1ts for 10 days per visit 
c _ intermittent for days over ^^^^ mos, 
d. - continuous for months 



20, 



19. Structure of Field Research (C) 

a* - one person throughout 

b. * • seri es of mmm _ mm sole persons 

c. ~cT team of J2_ persons 
team of persons 



Techniques of Qualitative Oata Collection (R) 21. Number of Field Workers (C) 10 



"a. document acquisition 

b * - questionnaire administration 

c highly-structured interviewing 

d. 1 semi -structured interviewing 

e. mmZm largely-unstructured interviewing 

f. - highly-structured observation 

g. - semi -structured observation 

h. HT 2g l largely-unstructured observation 
i. 



23, 



24. 



Types of Public Unit-specific Narratives (C) a. 

b. 
c. 
d. 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 



a. — anthropology 

b. education 

c * political science 
d. psychology 
a. ^i. sociology 

f - 3- TObliC adHSn-f^^nn 

none 

- brief vignettes 
IT chapter-length case studies 

<T book-length case studies 



Facts of Publication of Most Informative Study Document(s) 

Goettal, Jobart, Barnard A. Kaplan and Martin E« Orland (*ith the assistance of 
^p^/^S 10118 * Jr *> ^ Shfllla D * Huff)* A Comparison of SS2A Title I in- 
SJ^ht States. Syracuse, NT: 1 Syracuse TJiiversity" ResearcFcSJ^TJf^fS pp# 

* !? r A each of * he s^^ 5 qualitative field work *as also conducted (and case 

stu^y narratives written) within each of four dllustrati™ school districts* 

*> 4 



96 



PROJECT PROFILE 

Section I_ 

] - 10 -22 2. Project Nam e Case Stadiaa In Sciaaca Education 

3. Funding Agency HSF/bSB/OPI (Ccntrict No. 76-2U3u) 

4. Project Monitor a. Arlen QglHeksm ( p*y Telepnone 

b. Liad* Injfoon (3rd) Telephone 



5. - Research ffnjvergitr of ni inoia c Cfaw -. 7 , - I " « — IT 



2 «1 years 



Urban*, 2L 61001 7 

9. Total Budget $ 

10. Project^ rector, a Robert S. Stake Telephon e (217) 333-3770 

b. JackA.Ewlar, Jr. Telephon e (217) 333-4382 

Overall Research Design a. ^qualitative study(les) only ~ 

b. jeseparate qualitative and quantitative substudles 
c - — integrated qualitative and quantitative substudy(tes) 
tome of Distinct Substudles 13. Substudy 14. Principal Investigator 15. Appx 

ype "Budget 

a. Case Stadias fir ill) m p„, ► • a . , „ 

b. _V^dationSurW Z j-^f 

d! — $IZZ 

e. — _ 



11. 
12. 



Section Us Case Studies ' ~ 

16. Unit of Qualitative Study Hl^h School 17 . Mo . of Un1ts (c) 

18. Modes of Field Research (C) 19 . structure of Field Research 



,20. 



c. T Remittent foFl£ days' oyer if ons c T 22l,°V- S ° 1e pePSOns 

d. X continuous for jffcntk Sifr 5.' ? gS o°J ~ ES5 
Techniques of Qualitative Data Collection (R) 21. Number of F1 el dicers (C) n* 

!: ± SS!5ii?S , SgS i t™«cn 22 - 

c _l. highly-structured Interviewing " 6ld Workers (C) 



d. - semi -structured Interviewing a 1 , Bthm „ lM „ 

e. X largely-unstructured Interviewing J" 4 22SCl 0fly 

f. jZm highly-structured observation 9 b c ' -r SfStoJ $e w. 

g. - senl-structured observation "T SycSSiv 

?: ? Sly 

23. Types of Public Un1t-spec1f1c Narratives (C) 



24. 



a . _ none 

b. 3_ brief vignettes (10-20 pp.) 

c. IT chapter-length case studies 

d. JT_ book-length case studies 
Facts of Publication of Host Informative Study Oocument(s) : 

* One flelctaorker covered two sites. 
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PROJECT PROFILE 

Section I ^ 

i m 10 , _ „ Bvalaatica of Project Information Package Dissemination 

1. ID 2. Project Same and Implementation »uu««.c a 

3. Funding Agency 0S/0P3E (Contract Kb. 300°76-O^f^ / 

Project Monitor a. Judith Barnes Telephone (202) 215-319$ 

b< Telephone 

5. Research American Institutes for Research 6. Start 7/76 7 End 1 / 79 

Or9!,ni2ati0n 1791 Arastradaro Road 3. Duration years ~ 

Palo Alto, C A 5u302 , g. To tal Budget S 300 K 

10. Project Director a, Peggie Campean (AIR) Telephon e (lag) ^93-3550 

b. Joanne Binkley (am) Telephon e (103) 9U1-9550 

11. Overall Research Design a. .qualitative study(ies) only ' 

b- separate qualitative and quantitative substudles 

c. jsjntegrated qualitative and quantitative substudy(les) 

12. Name of Distinct Substudles 13. Substudy 14. Principal Investigator 15. Appx. 

Type 3udget 

a. laclamantation Case Stud ies Ql/Qn ma S SOQ K 

b. DisaeniaatiQn fa,^ Mies oi/on . " ha s ^ on k- 

J- : — - 5IZZ 

e. t 



Section IU Baplsreentation Case Studies 

16. Unit of Qualitative Stud y Sch. Dist. relaxant sch3.) 17. No. of Units (C) 18 
13. Modes of Field Research (C) 19. Structure of Field Research (C) 

*• 4- 4- v ! s ! ts ftr -S-* 1 ^ P ep v1s1t - one person throughout 

b - 1L. 4- v1 f 1ts for -L. per visit b. ^. series of sole persons 

c. intermittent for days over mos. c. 13" team of 2~~persons 

d. continuous for months d. team of _I persons 

20. Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) _6_ 

a. jj_ document acquisition 22. Major Disciplinary Ident1f1ca- 

b. - questionnaire administration tion of Field Workers (C) 

c. highly-structured Interviewing 

d. semi -structured interviewing a. - anthropology 

e. ^ largely-unstructured Interviewing b. - education 

f. - highly-structured observation c. "^"political science 
9- JL semi -structured observation d. T" psychology 

h. largely-unstructured observation e. m T m sociology 

*• ; f . — 

23. Types of Public Un1t-spec1f1c Narratives (C) a. x none 

b. brief vignettes 

C .-„ . chapter-length case studies 
d. _ book-length case studies 

24. Facts of Publication of Most Informative Study Document(s): 

Canpeau, Peggie, Joanne 3inkley, Peter Treadway, Judith Apoleby and Barbara 3 essay, 
£M ?P°rtt Evaluation of Project Information Package ' Dissem ination and 
Implementation. ?a!o Alto. CA: American institutes for Research, 15797X20 po. 

Not2 s This project nas the third in a series of three 0PB2-funded studies of PIPs. 
The first study was conducted by the^RXC Corp., the second by SRI Intnl. 
and SMC and this one by AIR -with an important subcontract to Hf!C. 
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PROJECT PROFILE 

Section I 

1. ID 11 2. Project Nam * A Study of EtoearleacBd-Based Career Education 

3. Funding Agency N3S/ay (Contract No« 1j0O^76*QL63) 

4. Project Monitor a. DaTidQoodHin Telephone (202) 2fli-6o70^ 

b. Telephone ^ 



5. Research The Huron Ihatitnte 6. Start 10 /76 7. End JL/80 

Organization 123 Mbont Auburn Street 8. Duration kg years 

Caafartdtee, Ml 02X38 J, Total Budget $ 600JC 

10. Project Director a, gleangr Farmp Telephon e (617) h91««gl£0 

b» Telephon e 

11. Overall Research Design a. t qualitative atudy(ies) only 

b. x s eparate qualitative and quantitative substudles 

c. Integrated qualitative and quantitative substudy(ies) 

12. Name of Distinct Substudles 13. Substudy 14, Principal Investigator 15, Appx. 

Type Budget 

a. SBC5 Prograa aarrej On Kk $ ho t 

b. SEk Policy Stntfar & NA $ 2h0 K 

C School IgpLenentaticn St acfar dl NA $ 320JC 

d. ' ZIZZZZZZZZ *_Z 

e. $ 

Section II: School Implementation Study 

15. Unit of Qualitative Stud y School 17. No. of Units (C)Jf5 



13. Modes of Field Research (C)' 19. Structure of Field Research (C) 

*• 41* i v ^ s1t ftr t^L days pir v1s1t *• — on * P ersort throughout 

b. 2-T visits for 52 days per visit b. - series of sole persons 

c. - intermittent for days over rtos. c. TjjT team of J2_ persons 

d. ^ continuous for i months d. team of persons 

20. Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) J3_ 

a. U document acquisition 22. Major Disciplinary Identifica- 

b. questionnaire administration tion of Field Workers (C) 

c. - highly-structured interviewing 

d. ~T~ semi -structured Interviewing a. m anthropology 

e. T largely-unstructured interviewing b. m education 

f. • highly-structured observation c. political science 

g. j£ semi -structured observation d. psychology 

h. «• largely-unstructured observation e. - sociology 

1 . ~ f. ~ 

23. Types of Public Unit-specific Narratives (C) a. x none 

b. — _ brief vignettes 

c. chapter-length case studies 

d. — _ book-length case studies 

24. Facts of Publication of Most Informative Study Oocument(s): 

Farrar, Eleanor, John £• DaSanctis and Drrid K« Cohen* "Views from Below: 

Implementation Research in Education," Teachers College 3a cord (Fall I98O), 
pp# 77-100* 
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PROJECT PROFILE 

Section I 

] ' 10 -35 — 2 - Project Name Program Consolidation and the State Role in Ti+l o 17 

3. Funding Agency OE/OPBS (Contract ftp. 300-77-0515) 

4. Project Monitor a. Penrose Jackson Telephone (Deceased) 

b. Anne Bezdak Weinb.eiaai» Telephone (202) 21£-8877 

5 ' SSStlon Iha g^Poration 6 . Start. JL/?L 7. End JJ79 

9 2at1 ° n 1700 «» Street 8 . Qurat1on {£ ^ ~ / ~ 

Santa Monica, ■ -OA QQu06 g. Total Budget $ 700 K 

10. Project Director a. IQlhrer Mclanghlin Telephon e (as) 393-OLll 

b v — Telephone 

11. Overall Research Design a. qualitative study(les) only 

b. — separate qualitative and quantitative substudles 

c ^Integrated qualitative and quantitative substudy(les) 

12. Name of Distinct Substudles 13. Substudy U. Principal Investigator 15. Appx 

— . Type Budget 

a. SSA Surrey (N »50) Qn NA > 

b. 1^ Survey (N «600T" 

c. g'iald Kesaar ch" 
d. 

• e. 

Section Ij: Field Research 
16. Unit of Qualitative Stud y State Education Agency u. No. of Units (C) 8 
18. Modes of Field Research (C) 19 . structure of Field ResearcTTcT 

b • 4- ^ 22S J!! t y l ptr a. _^ one person throughout 

d- ^ continuous- for ^months - — d .- ^" team. of persons 

20. Techniques of qualitative Data Collection (R) 21. Number of Field Workers (C) JO. 

b" - 1 - 22SJinIf2 1 J32:? ♦ ZZ ' Major disciplinary Identlflca- 
o. questionnaire administration tlon of Field Work*™ (r\** 

C highly-structured Interviewing U) 
2* "J" 5^;?^^ Interviewing a . - anthropology 

J' X J*2« 1 ^ n$t r uct « r «i Interviewing b . X education 

f. highly-structured observation c . 5~D0l1t1cal sei*nc» 

g. s semi-structured observation 5 "T PsycholJiv 

h. X largely-unstructured observation e T" sociology 

1 - : f. 

23. Types of Public Unit-specific Narratives (C) a. x none 

o. mmmm brief vignettes 

c chapter-length case studies 

<*• book-length case studies 

24. Facts of Publication of Most Informative Study Oocument(s): 

!M £2£5 1°*%*** "vE? *• M^UaghUa (with the assistance of Mlllicent Cox 

SdtS kaS^e SSSte, 1 '?? ^ d Gail Z " llcan ^ ft*™ Co nsolidation ' 
X^irT^Tir^ ~ — — ~ SaQti K0Qica > 018 TKr?CT ^oration/ 



Fialdwork was conducted both in the state capitol and in three school districts • 
^LlSoSers^ 011 * ^ 11 - trainad *** «*p«1enced social scientists 
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PROJECT PROFILE 

Section I, 

1 • ID 21 2. Project Name Vocational Bdncat.^ 

3. Funding Agency QB/OEat (Contract No. 3QO-77-031B) 

4. Project Monitor a. Dcrottar Sta ler Telephone (202) 2^-8877 

°* • Telephone 

5 - Sr^mtlon lMtitata8 f0r ReSWarffh 6 ' St * rt -i/-ZZ 7. End JjJ2_ 

*Z& Arastrmdero toed 8. Duration 1.3_ years 

Palo Alto, CA 9h302T o. To t«l Budget $ 9QU K 

Project Olrector a. Laurie Harrison Telephon e (lag) u93-3^0 

D • Tel ephone__ 

Overall Research Design a. .qualitative study(les) only "~" 

b. jcjeparate qualitative and quantitative substudles 
• c * «_Jntegrated qualitative and quantitative substudy(les) 
Name of Distinct Substudles 13. Substudy 14. Principal Investigator 15. Appx 

T yP* Budget 

$ 200 K 

$ 600 K 

$ 100 K 
% 



10 
11 

12 



a. 
b. 


Ere apiary Case Studies 
ramry liaU ^oilectioa 


oa 

<5n 


HA 


c. 


Literature rtavlew 


"HA 


MA 
ilk 


d. 








6. 









16. Unit of Qualitative Stud y Local Vqc. 3d. Program 17. No. of Un1ts ( C j 22 
18. Modes of Field Research (C) 1 9 . structure of Field Research"^ 

J'-^-i. J» r -3- ?y* P» v Js1t a. 12_ one person throughout 

c. intermittent for days over mos. c. team of Persons 

d. contl nuous for .Tenths — d . ^ of — JES 

20. Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) 

a. X acquisition 22. Major Disciplinary Ident1f1ca- 

b. questionnaire administration tlon of Field Workers (C) 
c - highly-structured Interviewing 

d. X semi -structured Interviewing a. - anthropology 

e. larg«1y-unstructured Interviewing b. V" education 

f. - hlgbly-structured observation c . T" political sdenc» 

g. X feml-structured observation d! TT psychology 

h. largely-unstructured observation e. - sociology 
f. i aconomics 

23. Types of Public Un1t-spec1f1c Narratives (C) a. none 

b. T2T brief vignettes (aptac. 20 pp.) 

c. .^chapter-length .^se studies 

d. book-length ca . studies 

24. Facts of Publication of Most Informative Study Oocument(s): 

Wh S?|* J 2? att8 ^^ J A3S0clAt<,s * 7 °P*^"*3. Education Equity Studr, Vol. 3- 
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Section I 



PROJECT PROFILE 



1 • ID }k 2 « Project Name A Stucfr o£ the R&D Utilisation Progran 
3, Funding Agency NE/DIP (Contract Mc+ hOO-78-C002) 



4. Project Monitor a . Must Ann itCLsap (1st) Telephone (617) 1j97-0S1lI 

b. John Egenneier (3rd) Telephone (202 ) 2$h-6Q50 

5. Research Abt Asaociatas lac. 6, Start 11/77 7. End 6/81 
Organization 5g WhM]ir strq<rt 8 ^ Ourat1on ^ ye&rs 

Cambridge, HA 6S138 T 0ta l Budget S .1.8 M 

10. Project 01 rector a. Karen Seashore Louis Telephon e (617) 956-1150 

b • ________ Tel ephon e 

11. Overall Research Design a. qualitative study(les) only 

b. _ ^separate qualitative and quantitative substudles 

c. x Integrated qualitative and quantitative substudy(ies) ' 

12. Name of Distinct Substudies 13. Substudy 14. Principal Investigator 15. Appx. 

Type Budget 

a. School Stucfer OlzQn M $ 1«2 H 

b. Pyojqct Study "~ Ql/qn KA $ Q«j ft 

c. Li nto n* Ageac .Study qn/qi m """" $ Q # 1 H 

<*. _ ZZZZZZIIZZZ $ 

$ 

Section II; School Study (Site Visit Sanpla Only)* 

16. Unit of Qualitative Study School 17, no. of Units (C) 51 

18. Modes of Field Research (C) 19. Structure of Field Research (C) 

JO JL. visits for 1_2 days per visit a. ^ one person throughout 

b * J2lJu. v1s1ts for J_ days per visit b. - series of sole persons 

c. Intermittent for ^ _ days over ^ _ roos. c, j£ ***** of 2 persons 

d * continuous for months d. team of persons 

20. Techniques of Qualitative Data Collection (R) 21. Number of Field Workers. (C) JJ_ 

a. J2_ document acquisition 22. Major Disciplinary Ideritifica- 

b. r_ questionnaire .administration tion of Field Workers (C) 

c. 3 highly-structured interviewing 

d. T_ semi -structured interviewing a. - anthropology 

e. largely-unstructured Interviewing b. education 

f. highly-structured observation c. - political science 

g. = semi -structured observation d. "T* psychology 

h. m m mm largely-unstructured observation e. T£ sociology 
— — f • 2 other 

23. Types of Public Un1t-spec1f1c Narratives (C) a. none 

b. JO brief vignettes 

c. ? chanter-length case studies 

d. mmJ * book-length case studies 

24. Facts of Publication of Most Infonatlve Study Document(s): 

a« Louis, Karen S«, Sheila Rosenblua and James Molitor* Itokjag R&D vith local 

Schools ? Strategies fbr School Improvement ♦ Cambridge, MA: Abt Associates Imu« 
X9BLV 280 pp. 

b« Louis, Karen S. "Sociologist as Sleuth: Integrating Methods fn the HDU Study," 
American Behavioral Scientists 1982, 26(1), 101.420. 

* The School Study used time data collection approaches. 2iB schools vere studied 
only through sita visits* 39 *«re studied only through intermittent field vork 
totaling 8 days during a 12 -north period* 3 schools vera studied both vays. 
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PROJECT PROFILE 



Section I 



1. 
3. 
4. 

o. 
10. 

n. 

12. 



10 *g • 2. Project Nans longitudinal Iaplapantation Stntty- of PL 9li-lii2 
Funding Agency OE/BBH than DE/CSS (Coatract Ko.' 300-78-0030) 



Project Monitor a. 

b. Martin Kanftnan 
Research SRI International 



(2nd) Telephone 
-(3rd) Telephone 



jgp2) h72-&52 



Organization 3,3 Aw# 



Menlo Park, OA 9uQ2g 



Project 01 rector a. Marian Stwns 
b. 



6. Start 1 / 78 7. End 10/ 82 

8. Duration years 

9. Total Budget $ 1*5 M 



Tel ephon e 859-3997 
Tel ephone 



Overall Research Oeslgn a. ^qualitative studyOes) only 

b. — separate qualitative and .quantitative substudles 

c — Integrated qualitative and quantitative substudy(les) 

Name of Distinct Substudles 13. Substudy 14. Principal Investigator 15. Appx. 

Type 

a. (ftona) 

b. ~~ — 

c. — ■ — 

d. — — 

e. — """ — 



Budget 



Section II 

16. Unit of Qualitative Study_ 
18. Modes of Field Research (C) 



$ 
$' 
$" 
$" 
$' 



School District 



17. No. of Units (C) 22 



20. 



•• J£<L visits for Ji. days per visit 

b. __6 U visits for _T days per visit 

c. Intermittent for days over nws. 

d. < __^ > continuous for months """"* 



19. Structure of Field Research (C) 



-iU. one person throughout 
D * - series of sole persons 

c. 22 team of 2 persons 

d. m m m team of persons 

Techniques of qualitative Data Collection (R) 21. Number of Field Workers (C) 8 



a. _2_ document acquisition 

JL, questionnaire adnlnl strati on 
c « -JL highly-structured Interviewing 
d. X_ seal -structured Interviewing 
•• _ZL largely-unstructured Interviewing 
*• JL highly-structured observation 
g. - semi -structured observation 
.X. largely-unstructured observation 



1 



23, 



24. 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

a. anthropology 

b< IE. education 

c - political science 

d. psychology 

e. sociology 

f. TT other 



Types of Public Unit-specific Narratives (C) a. none 

b. x 

brief vignettes 
c « ^L. chapter-length case studies 
d. book-length case studies 
Facts of Publication of Most Informative Study Document(s) : 

sta ^> ?* vid GrB «w and U David. Local Implementation of PL 9L-U2. 
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PROJECT PROFILE 



Section I 



1 • * a }£ [ 2. Project Name Th » Q*r*nr Intern Program Stu dy 
3. Funding Agency NISAI for POL (Contract No« 1j00-78-OO21) 

lu Project Monitor a. Hoard M» Leenlck (1st) Telephone 

b. BtgUl P» Antonopoloa (2nd) Telephone (202) 2^271 

5. Research RHC flwearch Corporation g. start 2 /78 7 Pad 8/81 

Or3a " 1at10n 22 SI GMdao tUil 3 . 0urJti rj5 y „„ 

Mountain Via*, CA~ 9li02$ 9. Total Budget $ 1.0 M 



10. Project Director a. Q. &at«n Tallmadge Telephon e (lag) 9itL-9$$0 

b « Telephone 

11. Overall Research Oeslgn a. qualitative study(ies) only 

&• _x n s eparata qualitative and quantitative substudles 

c Integrated qualitative and quantitative substudy(ies) 

12. Name of Distinct Substudles 13. Substudy 14. Principal Investigator 15. Appx. 

Type ' Budget 

a. Implementation Ql/Qn Peter Treadnay S 300 K 

b. Psychological Outcomes On ~G« Kasten TalLnadge ' ~* $ 300 K 
C. Lapl.-Otttcomas Relations Ql barld Fettaraan $ ' 550 T 

d. Other S imil a r Program Ql/Qn 71. Kaate a TalLaadge " $"T55"T 

e. 3 



Ql 


David r'ettaraan 


' Ql/Qn 


0. Aasten l»n™rf M 





15. Unit of qualitative Stud y 'Career Intern Program Site 17. No. of Units (C) li 




18. Modes of Field Research (C) yrsy 19 ' Structure of Field Research (C) 

a. _J*.JL visits for 10, days per visit/\ a. ^ one person throughout 

°» — • visits for days per visit b. - series of sole persons 

c L Intermittent for days over nws. c. team of 2" persons 

<*• «Jl continuous for months d. team of ~ persons 

20. Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) J_ 

a. 5 document acquisition 22. Major Disciplinary Ident1f1ca- 

°« JL questionnaire administration tlon of Field Workers (C) 

c. • highly-structured interviewing 

d. semi-structured interviewing a. 1 anthropology 

e. JT_ largely-unstructured interviewing b. "2"" education 

f. - highly-structured observation c. - political science 
9- a era1 -structured observation d. psychology 

'h. _2Z large! y-«unstructur*d observation e. - sociology 

*. • f. 

23. Types of Public Un1t-spec1f1c Narratives (C) a. - none 

b. J£2 brief vignettes (on many topics) 

c. TP chapter- length case studies (1 topic) 

d. book-length case studies 

24. Facts of Publication of Most Informative Study Document(s): 

a. Fettaraan, D.M. Study of the Caraar Intern Program * " Final report — Taak C: Program 

dynamics* Structure ^notion and interrelationships . Mountain Vleu", CAi"rmg 

flasearcn Corporation, 19ol. " 

b. Fettaraan, DJf, Bthnographio techniques in educational evaluation: An illustration. 

In A. VanFleat (Ed.) Anthropology of education : Kathode and applica tions. 
Journal of Thought (special topic edition) 1980, 15(3), 31^87 

c. Fettaraan, DA HLasdag the victimi -The problem of evaluation design and federal 
involvement and reinforcing vorld viens in education, Hunan Or»* ,198l,ljO(l ) ,67-77 . 
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PROJECT PROFILE 

Section I 

1 • 10 , 17 ,. 2. Project Nam e T»»cfa»r Corps Sraloatlcn Stndy 
3. Funding Agency QE/OEP (Contract No. 300-78-0289) 
4-. Project Monitor a. &gape Tuck«r 



Telephone (202 ) 2U5-8?Qo 



■ Telephone 

5. Research 3 HI Iat etna-tional c c * •> Q ,11 — T~ 

2 ^-2 9. Total Budget $ 5.0M 

10. Project Director a. Rldard, a« Marelano Telephon e «M) 8^611 

b * . , — Telephon e 

11. Overall Research Design a. ^.qualitative study(ies) only * ~ 

e* ^J^SJ"* 11 ??! 1 !? and ^»"«tat1ve subsidies 

c * — integrated qualitative and quantitative substudy(ies) 

12. Name of Distinct Subsidies 13. Substudy 14. Principal Investigator 15. A ppx. 

t , yP * 8udget 
a . Xaplaaw nsatlan Study I m T _ . 3 

J-iSSSPSE: i^" tosh : 

IS. Mt of 0j»Ht»t1vt Study Twehar Corps Pwinffc ; , 7 . of Unite (C) t, 

18. Mod* of Fl.ld R^rch (C) , 9 . S{nictup , of F ,„ d ResMr ^ 

o' -= &I — p,r v ] s l! »• -2. on* pmon throughout 

J. ^ contl*™ for * - • J; ± g g _ £«• 

20. Toctariqm. of W1t.ttv.0K.. C0l1«tfon (« 41 . Itabtr ,f F i, )d Horkw , ( C ) (, 

c. ^ highly-structured Interviewing 8 d WorlC<n (C) 

a. - sewl-structunsd Interviewing , i ,nth«,n«i««» 

Jt. highly-structured observation J — SffiVS scUnem 

g. - senrt-structur»d observation 4 — EycSloav 

k. S largely-unstructured observation e! X SSSogT 

23. Types of Public Unit-specific Narratives (C) a. nonT 

b - If brief vignettes (in Vol. I) 

JL chapter-length case studiesfrol. m 
d. _i bdok-length case studies ; 

24. Facts of Publication of Most Informative Study Oocument(s): 
a. ~ " • " - ■ 



b. 
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PROJECT PROFILE 

Section £ 

1 . ID 18 2. Project Name Youtbwork National Policy Study 

3. Funding Agency Touthwork Inc. (as a contractor to CQL/CflP) 

4. Project Monitor a. Lois-SIlin Data (1st) Telephone (202) 2Sk-6000 

b< ttlchael Laagaforr (7th) Telephone t 8o °) 

5. Research Corrmn University 6. Start _8 /78 7, End 9 / 80 

Organization N ^ s# College of Human Scology 8. Duration ja years ~"~ 

Ithaca, tt.Y. I2i8g3 9. Total Budget $ UP M 

10. Project Director a. Ray C. Rist ( Telephone (202) 275-0200 

b« ^ Tel ephone 

11. Overall Research Oeslgn a. x qualitative study(les) only 

Note : HITS vas imbedded b. separate qualitative and quantitative substudies 

^a^la^^r^^^rasi^of c. integrated qualitative and quantitative substudy(las) 

12. Name of 01 sti net Substudies 13. Substudy 14. Principal Investigator IS. Appx. 

Type Budget 

a. (None) - $ 

b. izzz — $ — 

$ 

d. g». ____ _____ $ 

■ ' S 

Section II 

16. Unit of Qualitative Study In-school Youth Project 17. No. of Units (C) 60 
18. Modes of Field Research (C) 19. Structure of Field Research (C) 

a- jz visits for days per visit a. 60^ one person throughout 

b. visits for days per visit b. jz_ series of sole persons 

c. -^Intermittent for days over roos. c. - team of persons 

d. 6CT continuous for 2^1nbnths(part^tlae) d. team of persons 

20. Techniques of Qualitative Data Collection (R) 21. Number of Field Workers (C) jSO. 

a. 6 document acquisition 22. Ma|pr Disciplinary Ident1f1ca- 

b* ^questionnaire administration tlon of Field Workers (C) 
c highly-structured Interviewing 

d» .1., semi -structured Interviewing a. 5 anthropology 

e. 3 1 argel y-unstructured Interviewing b. 3tT education 

f. «. i highly-structured observation c. - political science 

g. semi-structured observation d. IjjT psychology 



h. largely-unstructured observation e. To sociology 

1. ^ ■ f. 



23. Types of Public Un1t-$pec1f1c Narratives (C) a. g none 

b. brief vignettes 

c. chapter-length casa studies 

d. _ book-length case studies 

24. Facts of Publication of Most Informative Study Document(s): 

a* See entire issue of Children and Youth Sendees Review (Vol 2, No. 1, 1980) 

b* Bridging the Gap : Policy Strategie s for Earning and Leam-t^, Sage, 1981 
c« Confronting Youth Unemployment in the 1980s, Perganon Press, 1980. 

d. National Agendas and Local Initiatives ; Perspectives on Youth PnemLoyna nt 
Programs, SUKY PressTTsfeET' : ' " ' — 

106 




1. 

3. 

*. 

5. 



10. 



n. 



12. 



PROJECT PROFILE 

- u A Studj of DiMaadajition Efforts 
Project Name Supporting School T mara7mmirk 



Funding Agency 0E/3ED thro DE/JPS (Contract No. 300«78«Q<27^ 



Project Monitor a. Aim Btdaic ytinhainar 
b. 

Research The Networ k Inc. 
Organization 



Telephone (202) 2^5-8877 
Telephone 



290 South Main Stre*te 



Aadorer , Ilk 01610 



. 6. Start 10/78 7. End 10,/ 81 
. 8. Duration 3>0 y^ n 
9. Total Budget S 3*0 M 



Project 01 rector a. David Crandall (P.I.) 

b. CherLia Thocaaon (PJ).) 

Overall Research Design *. .qualitative study(les) on7y~ 

b * — separata qualitative and quantitative substudles 
c. reintegrated qualitative and quantitative substudy{1es) 



Tel ephon e (617) h 70-1080 
Tel ephon e (6I7) b70-lQ8o 



tome of Distinct Substudles 13. Substudy 14. Principal Investigator 

Type 



*♦ .School Sxrrrrmr 



b. Rroran 



C. Ertamal Xam^ 



.SL 



-9a. 



stain stodlaa 
Section II. school C*a« Stadlee 
16. Unit of Qualitative Stud y School 



field aaaUU 

Charlfls Thm^r^ 
Ronald ff^ln^ 



15. Appx. 
Budget 

S18J22EL 
* uOQK. 



S 

S-2SEBL 

S ?oor 



18. Modes of Field Research (C) 

*• visits for days per visit 

b ' !l5 1t5 f01 * B d«ys per visit 

c. intermittent fbr~8 , days over 6 nws. 
«• discontinuous for nonths """" 



20, 



17. No. of Units (C) 12 

19. Structure of Field Research (C) 
a. IL_ one person throughout 
b « -JL "rles of sole persons 

c. JT teen of _2_ persons 

d. teaa of persons 



Techniques of Qualitative Data Collection (R) 21. Number of Fteldlterkers (C) h 
* U document acquisition 

questionnaire administration 
highly-structured Interviewing 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 



i - Jl. stwf-structured Interviewing 
•• -1> »*rge1y-unstructured Interviewing 

f. - highly-structured observation 

g. Z£ semi -structured observation 

J. jz. largely-unstructured observation 

1 • 



23. 



24. 



Types of Public Un1t-spec1f1c Narratives (C) a. 

b. 



*♦ - anthropology 

b. m ^ m education 

c « - political science 

-J- Psychology 
t. jTj sociology 



non « 

— -1. brief vignettes 
c. _jT chapter-length case studies 
a. jL>ok-length case studies 
Facts of Publication of Host Informative Study Oocument(s)- 



9 
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Section 



1: 
3. 
4. 

5. 

10. 
11. 



ID 20 2. Project Name 



PROJECT PROFILE .< 

i 

k Study of Parental Iaovlveraent in 
Federal Education Programs 



Funding Agency Q5/OSD than DB/OEB (Contract No. 300-78-01j37) 
Project Monitor a. Daniel 0, Ozenne 



b.- Gerald Btirns 



Telephone (213) 689^23lig 
Telephone (202) 2)6JhOL 



Research 
Organization 



System Dgraloraant Corporation 
2500 Colorado Avenue . 



Santa Monica « Ci .90b06 



,6. Start 10/_7_8 7, End 7/ 81 
S. Duration 2.8 years 
9. Total Budget $ 2.0 M 



Project Director a. Raymond B. St wart 
. J. Ward wesiing 

p. Ralph j. taiacaeafl 



Tel ephon c (213) 8204j111 
Telephone (Z>3) 820-4031 



Overall Research Design a- qualitative study(les) only 

b- *_sep*r*te qualitative and quantitative substudies • 
c ' Integrated qualitative and quantitative substudy(ies) 



12. Name of Distinct Substudies 13, 



Site Study 

Federal Programs Su r vey 



Bubs*, 14. Principal investigator 
Type 

OX/Qn NA 

— nr 



IS, 



Appx. 
8udget 

S 1.5 M 

s 
s 



Section IX : site Study 



15. 
13; 



Unit of Qualitative Stud y Local Project* 



17. No. of Units (C) 57 



Mo v des of Field Research* (C), 
a. _ mZm visits for 'days per visit 



19. Structure of Field Research (C) 



20. 



- — — - . a. J&one P*«on throughout 

visits for days per visit b. series of sole persons 

c. intermittent for days over mos. c. • team of persons 

d. J£L continuous for months (J tiaeT d, team of ~ persons 

Techniques of Qualitative Data Collection (R) 21. Number o#-W^ workers (C) SL 

22. Ma-jor Disciplinary Identifica- 
tion of f )ld Workers (C) 



a. _k anthropology 
b* education 
c * ± political science 
d - ZEEl psychology 
e, ~F sociology 



f . "25" other 



23, 



24. 



none 



*• Jl, document acquisition 

b. . g questionnaire administration 

c. highly-structured; interviewing 
i. l ft g semi -structured interviewing 

1*SL largely-unstructured interviewing 

f . highly-structured! observation 

g. 5 seni -structured observation 

h. j largely-unstructured observation 

1 . ^ 

Types of Public Un1t-spec1f1c Narratives (C) 

b. - brief vignettes 

c. ^ chapter-1 ength case studies 

d. mmA book-length case studies 
Facts of Publication of Most Informative Study Docuaent(s): 

Lae, Dean R., J. Ward Keesling i Ralph J» Melaragno (Eds.). Parents and Faderal 
Educational Programs , Volune 7: Methodologies Employed in the Study" c*f Parental 
Involvement . Santa Honica. CA: System Development Coloration, 19817 115 pp. 

* A "local project" consisted of a central project office and two of the various schools 
participating In that project. The 57 projects were -associated with one of four 
federal programs 2 Follow Through, ESSA, Title I, Title VII. Cross-site analyses 
were conducted both vithin each of the four programs and across then. 
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Section I 



PROJECT PROFILE 



1. 
3. 
4. 

5. 



10. 



11. 



12. 



10 P 2. Project Name District Use of Infomating 



Funding Agency HIS/TL & OS/OPBS (Ccctract Ko. b 00-79-00611 
Project Monitor a. ^A^J^ffl flffifl "~ " 

b. Notramn 0614 (THE) 

Research 
Organization 



123 Mt> Aatcrn Street 
C*nfarldgt>, 02138 



Telephone J 

Telephone (202) 2fl-6271 



6. Start 9/79 7. End .8/ 82 
8. Duration 3.0 years 



Project Director a. Mary Kennedy 
b. 



9. Total Budget S $90 K 



Tel ephon e (617) u91-fli5u 
Tel ephone__ 



Overall Research Design a. ^qualitative study(les) only 

I' — S?I2Uf J"* 11 ??! 1 !? and ^"tltatlve substudles 
c * — intagrtttd qualitative and quantitative sucstudy(les) 
Name of Distinct Substudles 13. Substudy 14. Principal Investigator 15 Appx 



a. Stage I (Diatricta) 

b. stara rr ffiig^ cr^^ 
c stapw ttt [SjattaaiaJI 

a. 

e. " — — — 




HA. 



-Hi. 



Section II (St*?* Tt Districts) 

16. Unit of Qualitative Stud y School Di strict? 
18. 



Budget 

$ 313 K 
$ 217 K 
I .6QJS 



20. 



Modes of Field Research (C) 

J|. 2 visits for >g days per visit 
°« _JlJL visits for j£ ^ys ptr visit 
c « a Intermittent for days over 
d. - continuous for months — 



IMS. 



1^. No. of Units (C) 18 

19. Structure of Field Research (C) 

a. _2_one person throughout 
b * series of sole persons 

c. JUT team of persons 

d. - team of persons 



Techniques of Qualitative Data Collection (R) 21. Number of Fielder leers (C) 13 



JL. document acquisition 

b. questionnaire administration 

c. -highly-structured Interviewing 
,jS seal -structured Interviewing 
a& largely-unstructured interviewing 
-highly-structured observation 

r searf -structured observation 
. largely-unstructured observation 



d. 
e. 
f. 

9. 
h. 
1. 



23. 



24. 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

*• 4L. tnthropology 
b. 31 education 
c - — Z Political science 

d. psychology 

e. j» sociology 



Types of Public Un1t-spec1f1c Narratives (C) a. none ~ 

b. brief vignettes 

c. _ chapter-length case studies 

_ «• book-length case studies 

Facts of Publication of Most Informative Study Document(s): 



11 o 
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* Section I 



1. 
3. 
4. 



ID 22 



PROJECT PROFILE 

Case Studies of Iaterorgantratlsnal Arrangements 
2* Project Name for Knowledge Utilisation • 



Funding Agency NIB/DIP (Contract No. ljOQ-79-0062) 



Project Monitor a. Ward S. Mason 
b. 

Research 



Telephone (202) 251j-6(£0 
Telephone 



Abt Associates Inc* 



Organization i$a New Hampshire Ave, 

W^yMwffhnw | D»P. 20016 



6. Start 10/79 7, End 3/81 
8, Ouration 1*5 years 



10. 



11 



12, 



Project Director a., 
b. 



; 9, Total Budget $ 130 K 



Robert Tin 



Telephon e (202) 1j66«4i31i3 
Tel ephone_ 



Overall Research Design a. * qualitative study(ies) only 

b. separate qualitative and quantitative substudles 

c Integrated qualitative and quantitative substudy(les) 

Name of Distinct Substudles 13, Substudy 14. Principal Investigator 15, Appx. 

Type Budget 

a. (Hone) 

b. zzzzz zz z zzzzm^n 

C. , 

d. ~ ZZZZ ZIZZZZZIZ!~~ 

e. ~ 



$ 

s" 
$■ 



Section II 

15. Unit of Qualitative Stud y Iater-crg, Arrangement 
18. 



17, No, of Units (C) 3 



20. 



a. j2- one P*reon throughout 

b. - series of sole persons 

c. 3 team of persons 

d. team of persons 



Modes of Field Research (C) 19, Structure of Field Research (C) 

a- 3* 1 visits for 5 days per visit 

b. 3* x visits for T~ days per visit 

c. * intermittent for days over roos , 

d. continuous for months 

Techniques of Qualitative Oata Collection (R) 21. Number of Field Workers (C) J_ 

a. 1£ document acquisition 22. Major Disciplinary Identl flea- 

fa. - questionnaire administration - — - . - wu 

c. ~ highly-structured Interviewing 

d. 12 semi -structured Interviewing 

e. = largely-unstructured interviewing 

f . highly-structured observation 

g. - semi -structured observation 

h. «• largely-unstructured observation 
1. 



23, 



tlon of Field Workers (C) ** 

a. anthropology 

b. education 
c mm m m political science 

d. m j bm psychology 

e. m j tm sociology 

f • 1 econonriga 



Types of Public Unit-specific Narratives (C) a. none 

b. jg_ brief vignettes 
c * 2 ■ chapter-length case studies 
d. book-length case studies 
24. Facts of Publication of Most Informative Study Document(s) : 

Tin, Robert and Margaret Gwaltnejr. Organisations Collaborating to Improve 
Educational Practice . Washington, b.C: Abt Associates Inc./Tsa, 135 pp# 

* The initial visit of $ days was the priaarjr, source of data. The secend visit of 
3 days was for the purpose of verifying the conclusions of the case study .narratives, 
liajor enphasis on the use of wen-trained and experienced social scientists as 
field workers. 
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PROJECT PROFILE 

Section I 



1. 
3. 



10 JL__ 2. Project Name ^^^^S^^n^^ *» 

4. Project Monitor a. Wax* S, uum Tel ^hone (202) 2^0 

j> . u b * Telephone 

ftSStlw f n TrZ TT ** 6 * SUrt **JL 7. Endj^ 

Knowledge Transfer Bistltata _ 8. Duration 1_7 years 

10. Project Doctor j.^^^ Telephon e ( 202 T^ L*u 

b. Michael Telephon e 

Overall Research Design a. ^qualitative study(ies) only ! 

J' — in2H!f J"* 1 ^! 1 !? and *»"«t»t1ve subsidies 
C * — 1nt «9rat«i qualitative and quantitative substudy(les) 
Name of Distinct Subsidies 13.jSubst.dy 14. Principal rnvestlg.tor 15. 

a. (Nona) B ^ et 

b. — 1 . $ 



11. 
12. 



c ' . $ 

d. _ . $' 

e. 1 " — — , . f 



Section II 

± j£ »M!S £ •* ft S %n J- 4- »it 

c. 3 7nt.™itt.nt for - doyTow mo: J - — VP" J" — M,e I*™" 

1- 3 continuous for _Tb£te ° ,W — ""• «• ^- *__ £ _ P«« 

20. W,. of Qunlttutlv. Dot, MM. (R) a. Bu .^ of „m^^L * 

•• -2- ^cumtnt acquisition ' « ll i 1 _ IM . J1J - i - 

b. ^ questionnaire ad»lS?strat1on *' ^ Jfifl'SS Id fS f 1ca ' 

c. _X highly-structured Interviewing ° f " ,1d Horkers (C) 

d. ^ seal-structured Interviewing , - _nthiw«i„m. 
-S. J»rgtly-unstructurea Interviewing J "S" J3iSSbi, W 

f. highly-structured observation ? T SiSir!? 

?• -X jean-structured observation 5* ~"T £ll£S] sc1ence 

J. largely^nstructured observation £ "f sIcTJloof 

1 • JJL logs of activ ities and etartacta ?. 57 

23. Types of Public Unit-specific Narratives (C) a. 



TT- nOI,t 

b. IT, brief .vignettes (the "serials") 

c. __i_ chapter-length case studies 



- d " —1- book-length case studies 

24. Facts of Publication of Host Informative Study Documents) • 



l 1 

X -A. i 
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Section I_ 
ID 2li 



1 . 

J * 

4. 



PROJECT PROFILE 

Case Studies of Interorganiaational. Arrangements for 
Project Name Knowledge Utilisation: CTniTersity/School/ConnnunitT Co llabs* 



Funding Agency NIS/DIP (Contract No« N33- 79-O06M 

Project Monitor a . J^ardSjJ&son Telephone (202) 2$1j-60$0 

b« Telephone 

Research 



TER Associates Inc« 



Organization 385 aiot 



Newton^ Kfc 



6- Start 10/79 7. End jW 81 
3. Duration 1*6 years 
9, Total Budget S 100 K 



10, 



n 



12. 



Project Director a. Robert Chin 
b. 



Tel ephon e (617) 969-06?! 
Tel ephone 



Overall Research Design a. x qualitative study(ies) only 

b. separate qualitative and quantitative substudies 

c. integrated qualitative and quantitative substudy(ies) 

Name of Distinct Substudies 13. Substudy 14, Principal Investigator 15. Appx. 

Type Budget 

a-, (None) _____ 5 

b. ; ZZZIZIZZZZI S 

c. S 

d. : ZIZZZZZIZZ s 

e. ; $ 



Section 

16. Unit of Qualitative Study_ 
18, Modes of Field Research (C) 



Event/Project* 



17. No. of Units (C) 12 



2Q. 



19. Structure of Field Research (C) 

*• _____ visits for days per visit a. 12^ one person throughout 

b. - visits for days per visit b. series of sole persons 

c * ±__ Intermittent for days over mos. c. mSmm team of persons 

d. continuous for months d. team of m persons 

Techniques of Qualitative Oata Collection (R) 21. Number of Field Workers (C) h 



a. _2_> document acquisition 

b. jm^ questionnaire administration 

c. highly-structured interviewing 

d. 1 semi -structured interviewing 

e. - 1 argel y-unstructured Interviewing 

f . jm_ highly-structured observation 

g. _2_ semi -structured observation 

h. largely-unstructured observation 
1. 

23. Types of Public Unit-specific Narratives (<C) 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

a. * u anthropology 

b. iT education 

c. - political^ science 

d. psychology 

e. 2 sociology 
f. 



a. mmmam none 

_2L brief vignettes 
c chapter-length case studies 
d. book-length case studies 



24. Facts of Publication of Host Informative Study Oocument(s): 

TDR Associates Inc* Case Stadias of Three Urban University - School Collaborations 

Mandated for the I mpo te p ent p r^^^^^Ti^^^ n^^ I & TT. 

Nawton/'MA: iSPauthor, I9BT,TI5 pp. & 271 pp." ' ~~ ~ ~ 



* The "event/project" was the most basic unit of qualitative study. ?or purposes of 
analysis separate cross-unit syntheses were conducted within each of 3 college/school 
collaborates and within the single schQcOJ district associated with them* 
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PROJECT PROFILE 
A DMcripfeiTi Stady of Significant 




10. 



n. 



12. 



San Frmndaeo. Ck 910.03 



Project Director t. William Tlfcanagf 
b. 



5. Start _? / 80 7. End ? / 8? 

8. Duration years 

9. Total 8udget $ 2-7 M 

Tel ephon e 565-3000 

_ Telephone^ 

Overall Research Design a. _qual1tat1v« study(ies) only ~~ 

I' x"5!EII!!!?J utl1 w! 1 !? and substudles 

c. ^integrated qualitative and quantitative atetudy to* 

Name of Distinct Subsidies 13. Substudy 14. Principal Investlgstor 15. Appx. 
a. Tear 1 qi/Qp Hi. 



b. Year 2 



c rear 1 
d. 
e. 



Ql»Qn 



UT 



Budget 
$ 1,1 M 

in 

$ 



Section _H (Xaar 1) 

16. Unit of qualitative Stud y Btbno-liagulatio Qrocp 

18. Nodes of Field Research (C) , 19. Structure of F1.ld R W «rcMcT 



17. Mo. of Units (C) 6 



20. 



-S visits for days per visit 

°- . visits for days per visit 

c. - Intermittent for days over <nos 

d. ^continuous for _J~obnths*** — ' 

Techniques of qualitative Data Collation (R) 21. Number of Fleld^rk.rs (C) 2b 
* 5 document acquisition — 



a. one parson throughout 
b * -i- "rln of sole persons 

c. _6_ teas of jwfpersons 

d. - team of persons 



a. -,..„.„.„.. 

b. a^estlonnalre'adiinlstratlon 

c. - highly-structured Interviewing 

d. U seal-structural Interviewing 

a. 6 largely-unstructured Interviewing 
f ' -i. Highly-structured observation 
9* 522. semi -structured observation 
I, Z£ largely-unstructured observation 

1 • 



23. 



24. 



22. Major Disciplinary Identifica- 
tion of Field Workers (C) 

*• m Zm anthropology 
b. ji. education 
c ^i. political science 
d. jj. psychology 
t. «• sociology 

f. arm 



Types of Public Un1t-sptc1f1c Narratives (C) a. none 

b. ^ brief vignettes 

c. -chapter-length case studies 

d. book-length case studies** 
Facts of Publication of Host Informative- Study Docunent(s) • 



*Tbe Tear 2 study included two additional sitae. 
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APPENDIX B. EXPLANATORY NOTES FOR THE PROJECT PROFILES 

Presented below are a series of notes helpful in 
understanding the raultisite qualitative project profiles 
?or S ^ d -i n A ? pen * ix A « These notes elaborate the short titles 
£JS ??V te ? foUnd °? the P roflle sheets and offer definitions 
'.or all key terms used on those sheets, in addition they 
clarify those instances in which the numerical "score" for a 

S2 X ??J?f Xt f B r ?? res ? n ^ s a count (indicated by a «c- after the 
item title— cf., item 18) and those in which it represents a 
rank (indicated by an -R- after the item title-cfT, item 20) 



t Section I of each profile presents information about a 
project irrespective of its substudy organization or the various 
units of qualitative study it may contain. In Section II 
descriptive data are presented for either the. entire project (if 
it consists of a single qualitative study) or for only a major 

s^ 1X SdieI). SUbStUdy Uf the Pr ° jeCt C ° n ^ ists ° f twS or more 

SECTION I. PROJECT SPECIFIC DATA 

1. ID 

^™\£ 0min ^ C0 ? e number f°f each project, running in sequence 
from the earliest funded (01) to the latest funded (25) project. 

2. Project Natme 

* 

T ? e n f me a . th ! P r °J ect is most widely known by. m general ' 
is not the formal project title associated with the 
contract document. 

3. Funding Agency 

The agency most directly responsible for the funding and 
monitoring of the research. (See Appendix A, Table A-l f or a 
glossary of funding agency abbreviations.) 

4 « Project Monitor 

The federal official most directly responsible for 
overseeing the technical aspects of the research, if there was 
more^than one monitor, unless otherwise noted the first one if 
reported on line "a- and the last on line "b'.» 

5. Research Organization 

The organization awarded the contract (or occasionally 
subcontract) to carry out the research. sionaiiy 
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6. Start Date 



The month and year in which the project began. 

7. End Date 

The month and year in which the project (including all 
follow-ons and refundings) officially terminated or (in the case 
of ongoing projects) is due to terminate. 

8. Duration 

The duration (or expected duration) of the project in years. 

9. Total Budget 

The project's total budget/ including all follow-ons and 
refundings. The abbreviation "K" signifies thousands and "M" 
millions. * 

10. Pro ject Director 

The person within the research organization (item 5) most 
responsible for the technical direction of the project. If 
there was more than one such person during the life of the 
project, unless otherwise noted the first incumbent is reported 
on line "a" and the last on line "b." 

11. Overall Research Design 

This item reports the relationship (if any) of the multisite 
qualitative activities within the project to other project 
activities. Qualitative studies are those involving on-site 
field work consisting of document acquisition/ interview or 
observation. They may or may not involve the preparation of 
site-specific narratives (e»g., case studies) for each site. 
"Separate qualitative and quantitative studies 11 generally have 
distinct data bases and reports. "Integrated" qualitative and 
quantitative studies do not. 

12 v Name of Distinct Substudies 

A brief descriptive name for each distinct substudy. 

13. Substudy Type 

Substudies are either primarily qualitative (Ql)/ primarily 
quantitative (Qn)/ a blend of qualitative and quantitative with 
the emphasis on the former (Ql/Qn) , a blend of qualitative and 
quantitative with # the emphasis on the latter (Qn/Ql) f or an 
approximately equal blend of both (Ql=Qn). 



11G 



14. Principal Investigator 



The name of the one person on the project (other than the 
project director) who was most responsible for the technical 
direction of 'the subatudy. 

15* Approximate Budget 

The approximate budget (including follow-ons and refundings) 

525" ^ h f,? ubstudv - ^ e abbreviation «K" signifies thousands and 
m minions ♦ 



SEgTION II. DATA SPECIFIC TO A SINGLE UNIT OF QUALITATIVE STUDY 

„_v If the project contains no distinct substudies (see item 
12 J section II presents information about the entire project 
If, however, the project contains distinct substudies section II 
reports on only the designated gualitative (or primarily 
qualitative) substudy. pxrimarixy 

16. Unit of Qualitative Study 

The unit under study by gualitative means. Units are- 
typically school districts, projects within school districts, 
schools, or projects within schools. Often the term research 
'site'* is used as a synonym for "unit." 

17. Number of Units 

The total number of units (i.e., research sites) under 
investigation by the study/substudy . 

18. Mode(s) of Field Research (counts of units) 

This item distributes the units reported in item 17 
according to three modes of field research varying in the 
temporal extensiveness of on- site presence. A -visit" involves 
«L m « re 14 cow^tive days on .site, it is usually made by 

persons living away from the site. "Intermittent" field research 
involves more frequent contact, generally by someone living 
within a day's commuting distance from the site and who is 
involved in research at that site on an on-going basis. 
Continuous" field research involves doing at least one-half 

SJI rS ^ arCh ^ SaCh Site ' generally by persons residing at the 
site. -The numbers presented are counts of units, and (unless 

item'Tf 6 ed) SUm t0 thS t0tal nUmber of rait Sported in 
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19. Structure of Field Research (counts of units) 

This item distributes the units reported in item 17 
according to different staffing arrangements. The numbers 
presented are counts of units and (unless otherwise noted) sum 
to the total number of units. 

20. Techniques of Qualitative Data Collection (ranks) 

This items presents a rank ordering of the degree of 
"importance" within the .study/substudy of various techniques for 
the collection of qualitative data. The emphasis here is on 
data collection techniques which were built into the multisite 
study design and thus were intended to be utilized rather 
uniformly across all sites / as distinct from techniques which 
may have been employed at the discretion of field workers 
responsible for only some of the sites. Importance is 
considered in terms of the centrality of a given technique to 
the public products of the study/ not in terms of how the budget 
was allocated. "Highly structured" techniques impose oji the 
field workers a detailed set of stimuli to persue (e.g./ issues 
to be addressed/ phenomena to be observed/ questions to be 
asked) as, well as predefined sets of response alternatives 
(e.g./ behavioral options/ precoded answers), "Semi- structured 11 
techniques impose predefined stimuli/ but not predefined 
response options. "Largely unstructured" techniques impose 
neither detailed stimuli nor response options / although they 
generally imply some broad priorities regarding the classes of 
stimuli and of responses to be explored. 

In general both highly structured and semi-structured 
techniques imply the existence of checklists or schedules of 
things for the field workers to ask about, or observe/ while 
unstructured techniques do not. Highly structured techniques 
imply the coding of data according to predefined categories 
while the data are being collected / whereas semi-st'ructured and 
unstructued techniques do not. The niwtbers presented in item 20 
are ranks for the various techniques in terms of their 
importance/ with "1" signifying the greatest importance. 
Techniques that were seldom employed within a particular 
study/substudy have been left blank (i.e./ unranked) . 

21. Number of Field Workers 

This is the total number of persons within the 
study/substudy who had major responsibility for on-site data 
collection. 
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22 • Ma joy, Disciplinary Identification of Field Workers 
(counts of field workers) 

This item distributes the total number of field workers 
reported m item 21 according to their major disciplinary 
identification. Xn most cases such identification is the field 
in which a person holds his or her highest degree, but it could 
be a different field if a shift in disciplinary interest has 
occurred suosequent to the awarding of the highest degree. The 
numbers presented. sum to the total number of field workers as 
reported m item 21. 

23, Tvjoe of Public Unit-specific Narratives (counts of sites) 

This item presents the emphasis within the study/sub study 
?£?!lrJ£S P re J entat i°? of £Hblic narratives (i.e., narratives 
intended to be read by persons beyond those working on the 
iSKLSf sp 5 C i flc sites. -Brief vignettes" are descriptive 
20 SSES abs * racts or Profiles of no more than approximatel y 
20 double spaced pages. "Chapter-length case studies" are 

a hS r °?^ a - el ^-2°" 100 Page ? in length and Provide a sense of what 
tne site is like as a social or cultural system . "Book-length 

2™ St * dleS ? re over 100 pages in lenghtind-provide a deeper 
sense of social or cultural system. The numbers presented 
represent the number of sites for which each type of public 
narrative was prepared. They do not necessaryily sum to the 
total number of sites, since more than one type of public 
narrative could be prepared for some (or even all) sites. In 
addition there can be a "nesting" of narratives, with 
chapter-lenght case studies (e.g., of schools) being located 
districts)?" 65 boo *- lei *Jth case studies (e.g., of school 

24. Facts of Publication of Most Informative study Document (s) 

The document (s) produced by the study/substudy which is 
(are) most informative about its multisite qualitative findinas 
and how those findings were produced. In many instance s thesf 
documents contain references to other (more specialized? 
study/substudy documents. ' 
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APPENDIX C. SUPPLEMENTARY ^TABLES FOR PART II 



Presented below are ten tables prepared in exploring 
the inter-issue patterns discussed in Part II. Table C-l 
presents the marginal frequencies for each of the four 
design-issue variables: 1) the predominant data collection 
approach, 2) the number of sites, 3) the degree of on-site 
presence, and 4) the analytic emphasis of report narrative. 
Tables C-2 thru C-7 display the six resulting two-variable 
associations. In only one instance (the number of sites by 
the analytic emphasis of report narrative) was a 
statistically significant association observed (see Table 
C~6). 

In order to explore for possible higher-order patterns 
of associations, Table C-6 was used to define five 
distinguishable zero-order patterns, labeled A, B, c, iS and 
E (see Table C-10). However, when these five patterns were 
related to the degree of on-site presence (Table C-8) and 
the predominant data collection approach (Table C-9), no 
statistically significant higher-order associations were 
observed. 



\ 
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Table C-l. Distribution of a Major Qualitative Study within 25 

Federally Funded Policy Research Projects on each of Four 
Design Variables. 



Design Variable 



Number of 
Studies 



A. The predominant data collection approach: 

Primarily semi-structured 
Semi-structured with some unstructured 
Primarily unstructured 



14 
6 
5 



B. The number of sites being studied: 

Three thru six 

Eight thru twenty- two 

Thirty thru sixty 



7 
13 
5 



C. The degree of on-site presence: 

One or two short visits 10 

Several intermittent visits 7 

Many repeated visits or continuous presence 8 



D. Analytic emphasis of report narrative: 

Primarily site-specific 12 

Primarily cross-site with some site-specific 3 

Exclusively cross-site 10 



12 



•> 1 



9 
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C-2. Joint Distribution of the Degree of On-site Presence and 
the Number of Sites 



Degree of 
On-site Presence 



3-6 



Number of sites 
8-22 



30-60 



One or two 
short visits 

Several inter- 
mittent visits 

Repeated visits or 
continuous presence 



2 
1 



5 
6 
2 



3 
0 
2 



Chi-square = 6.7 (df a 4) 



Table C-3. Joint Distribution of the Degree of On-site Presence and 
the Predominant Data Collection Approach 



Degree of 
On-site Presence 



Predominant Data Collection Approac h 



Primarily 
Semi-structured 



Largely 
Semi-structured 
with some 
Unstructured 



Primarily 
Unstructured 



One or tvo 
short visits 

Several inter- 
mittent visits 

Many Repeated visits or 
continuous presence 



7 
5 
2 



2 
1 
3 



1 
1 
3 



Chi-square =4.8 (df a 4) 



1 *>«: 
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Table Joint Distribution of the Degree of On-site Presence and 

the Emphasis of Report Narrative 



Degree of 
On-site Presence 



Analytic Emphasis of Report Narrative 



i Primarily 
Site-specific 



Largely C-s 
with some S-s 



Primarily 
Cross-site 



^ne or two 

short visits 3 



Several inter- 
mittent visits 4 

Repeated visits or 

continuous presence 5 



5 
3 
2 



Chi-square =3,2 (df = 4) 



Table C-5. 



Joint Distribution of the Number of Sites and the 
Predominant Data Collection Approach 



Number of 
Sites 



Predominant Data Collection Approach 



Primarily 
Semi-structured 



Largely 
Semi-structured 
with some 
Unstructured 



Primarily 
Unstructured 



3- 6 
8-22 
30-60 



5 
7 



1 
3 



1 
3 
1 



Chi-square =,1.5 (df = 4) 



9 
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Tab le c-6. ^i^trib^io^of the Muaber of Sites and the Emphasis 



Number of 



Analytic Emphasis of Report Narrative 



Sites Primarily Largely c^s Primarily 

Site-specific with some S-s Cross-site 

3 ~ 6 6 1 0 

8-22 6 i 6 

30-60 o 1 4 

Chi-square = 9.9 (df = 4) 



Table C-7. Joint Distribution of the Predominant Data Collection 
Approach and the Emphasis of Report Narrative 



Predominant 
Data. Collection 



Analytic Emphasis of Report Narrative 



ERIC 



Approach -Primaril y Largely c^s Primarily 

K Site-specific with some s-s Cross-site 

Primarily 

semi-structured 8 2 . 4 

Largely semi-struct. 

some unstructured 0 \ 

Primarily 

unstructured 4 n 



Chi-square ■ 8.7 (df = 4) 
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Table C-8. Joint Distribution of the Degree of On-site Presence and 
NS/RN Design Pattern 



Degree of 
On-site Presence 



NS/RN Design Pattern* 
BCD 



One or two 
short visits 

Several inter- 
mittent visits 

Repeated visits or 
continuous presence 



1 
3 



3 

0 

2 



Chi-square = 10.2 (df = 8) 

* See Table C-10 for the operational definition of the five design 
patterns. 



Table C-9. 



Joint Distribution of the Predominant Data Collection 
Approach and NS/RN Design Pattern 



Predominant 
Data Collection 
Approach 



A . 



NS/RN Design Pattern* 
BCD 



Primarily 

semi-structured 

Largely semi-struct. 
some unstructured 

Primarily 

unstructured 



ERLC 



Chi-square = 12*8 (df = 8) 

*' See Table C-10 for the operational definition of the five design 
. patterns. * 

. 1 3 0 
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Table C-10. Definition of the Number of Sites/Analytic Emphasis of 
Report Narrative (NS/RN) Design Patterns A, b7 C, D & E 



Number of 
Sites 



3- 6 
8-22 



Analytic Emphasis of Report Narrative 

Primaril y Largely C^s Primarily 

Site-specific with some S-s Cross-site 



B 



D D 
30-60 . E E 
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APPENDIX D. ANNOTATED BIBLIOGRAPHY FOR PART III 



The Rural Experimental Schools ( RES ) study 



A * Herr i Qtt ' R«E. 'Federal initiatives and rural school improvement t 
Findings from the Experimental Schools Program . Cambridge, MA: 
Abt Associates Inc., 1980, 36p. (ERIC Document Reproduction 
Service No. ED 192 961) 



An overview of the design and substances of the RES Study. 
Presents an annotated bibliography of the eight nonstandard 
book-lenght case studies and an informal cross-case synthesis of 
their major findings. 



B. Herriott, R.E. & Gross, N. (Eds.). The dynamics of planned 
educational change ; Case studies and analyses . Berkeley, CA: 
McCutchan, 1979, 411p. 

A presentation of five semi-standard chapter-lenght case 
studies along with the nonstandard cross-case analyses of five 
external experts. 



C. Herriott, R.E. Tensions in research design and implementation: 
The Rural Experimental Schools Study. American Behaviora l 
Scientist , 1982, 26(1), 23-44. . 3^ 

An account by the RES Study director of tensions in the 
process of its design and implemention. Focuses heavily on 
isssues involving the RES case studies and the interaction of 
qualitative and quantitative methodologies. 



The Experienced Based-Career Education (EBCE) Study 



D. Farrar, E., DeSanctis, J.E. & Cowden, P. The walls within : Work, 
experience and school reform. Cambridge, MA: The Huron 
Institute, 1980, 20 lp. (ERIC Document Reproduction Service No. 
.ED 203 193) 

<* The first of a two-volume final report from the EBCE Study. 
Lucid qualitative description and analysis of schools as social 
systems, of the process of EBCE Implementation, and of its early 
outcomes . 
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E. DeSanctiS/ J.E. & Stix, 3.A.' Federal policy and local effects : 

The implementation and institutionalization of Experienced-Based 
Career Education . Cambridge/ MA: The Huron Institute, 1981/ 
125p. (ERIC Document Reproduction Service No. ED 20.3 192) 

The second final report volume. A mixture of qualitive and 
quantitative analysis, focusing primarily on the impact of 
external forces on local EBCE implementation and 
institutionalization. 



F. Farrar/ E. / # DeSanctiS/ J.E. & Cohen/ D.K. Vievsf f rom below: 

Implementation research in education. .Teachers College Record/ 
1980/ 81, 77-100. 

An interpretation of the qualitative data in the light of 
three perspectives on implementation as a social phenomenon: 
implementation as a center-to-periphery process, implementation 
as a bilateral process/ and implementation as evolution. 



The Career Intern Program (CIP) Study 



G. Tallmadge/ G.K. / Treadway, P.G. & Fetterman, D.M. Study of 

career intern program: Summary report . Mountain View, CA: RMC 
Research Corp./ 1981. (ERIC Document Reproduction Service No. ED 
. 206 840) 

Summarizes results of all aspects of the CIP evaluation. 
Includes quantitative assessment of program outcomes, case 
histories of the four sites and analysis of program elements 
conducive to positive results based on qualitative data. 



H. Fetterman, D.M. Study of the Career Interh Program: Final Report 
Task C. Program dynamics : Structure/ function and 
interrelationships . Mountain View, CA: RMC Corp./ 1981/ 314 p. 
(ERIC Document Reproduction Service No. ED 206-843) 

Provides qualitative description of the career Intern Program 
as it was implemented in four sites. Identifies program 
characteristics that facilitate and impede positive outcomes. 
Extensive use of field notes for both descriptive and analytic 
purposes. Presents chapter length case "studies on selected 
topics. 



I. r Fetterman, D.M. Ethnography in educational research: The 
dynamics of diffusion/ Educational Researcher / 1982/ 11(3)/ 
17-22/ 29. 

An examination of tensions within the CIP Study from a 
perspective emphasizing the interaction of two sociocultural 
systems, those associated with "ethnographic * ,and 11 quantitative 11 
research methodologies* Contains citations to other Cl£-related 
methodological 'discussions . 1 9 Q 



The Parental Involvement: ( PI ) Study 



J. The four substantive final report volumes are: 

Robbins / A.E. & Dingier/ D. Parents and federal education 
procrrams # Vol * 3t ESS A * Santa Monica/ CA: System Development 
Corp*/ 1981. (ERIC Document Reproduction Service No. ED 218 785) 

Cadena-MunoZ/ R* & Keesling/ J.W* Parents and federal 
education programs , Vol . 4: Title VII. Santa Monica/ CA: System 
Development Corp*/ 1981.-TERIC Document Reproduction Service No. 
ED 218 786) 

Smith/ A.G. & Nernberg/ S. Parents and federal education 
programs / Vol . S>: Follov Through . Santa Monica/ CA: Systom 
Development Corp./ 1981. (ERIC Document Reproduction Service No. 
ED 218 787) 

Melaragno/ R. / Lyons / P. & Sparks / M. Parents and federal 
education programs / Vol . 6: Title I. Santa Monica/ CA: System 
Development Corp./ 1981. TERIC Document Reproduction Service No. 
ED 218 783) 



Each volume uses qualitative data to describe five dimensions 
of parent involvement in federal programs and to identify causes 
and consequences of such involvement. Heavy emphasis on the use 
of charts comparing sites within a program on a series of 
dimensions derived from the field notes. Some narrative 
description through the selective use of vignettes. Each volume 
deals with from 12 to 16 districts participating in one of the 
four federal programs. 



K. Ijae/ D.R./ Keesling/ J*W. & Melaragno/ R.J. (Eds.).. Parents and 
federal education programs / Vol . Methodologies employed in 
the Study of parental involvementT Santa Monica/ CA: System • 
Development Corp./ 1981/ 57 p. plus appendixes. (ERIC 'Document 
Reproduction Service No. ED 218 789) 

Describes the methodologies employed for both the survey and 
qualitative portions of the study. A. special chapter deals with 
the strenghts and shortcomings of the qualitative methodologies 
employed in the study. 

L. Smith/ A.G. & Robbins/ A.E. Structured ethnography: The Study of 
Parental Involvement. American Behavioral Scientist # 1982/ 
26(1)/ 45-61. 

Describes in detail the procedures used to collect/ reduce/ 
analyze and report the PI Study data. Examines some of the 
tensions faced in the study's design and implementation. 
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M. Huberman/ A.M. & Miles/ M.B. Kow school improvement works : A 
field study of 12 sites . And oyer/ MA: The Network/ Inc./ 1982/ 
525p. 

Presents in great detail the methodology and findings of the 
DESSI qualitative study of school improvement efforts at the 
local level. 



N. Huberman, A.M. & Crandall/ D.P. Fitting words to numbers: 

Multisite/multimethod research in educational dissemination. 
American Behavioral- Scientist / 1982/ 26(1)/ 62-83. 

Examines the DESSI qualitative study of school improvement 
efforts in the light of the larger project of which it was a 
part. Focuses on issues. of validity, reliability/ and 
completeness. Presents an overview of the DESSI Study's 
innovative approaches to the reduction and analysis of 
qualitative data. 



0. Huberman/ A.M. & Miles / M.B. Drawing valid meaning^from 

qualitative data: Some techniques of data reduction and display. 
Quality and Quantity / 1983/ 1,7/ forthcoming. 

Considers the utility of qualitative methods in efforts to 
produce useful generalizations. Presents illustrative examples 
of the DESSI Study 1 s approach to the reduction/ display and 
cross-site analysis of qualitative data. 
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